Independent Categorization Task for Exchanges -- 2

I am an independent researcher running a small reliability study. I am seeking two people to work independently, plus one reserve, to apply a written six-category codebook to 20 short written exchanges. Each exchange contains a claim, a challenge and a reply. You will read the reply and select exactly one category using the supplied instructions.

The task needs careful reading and judgement under a written rule. Subject expertise, programming, a publication record and experience of AI evaluation are not required.

Time and pay
The main task is expected to take about 60–90 minutes in one sitting. Pay is A$50 per hour, calculated from time worked, with a maximum of A$75 for the main task per activated coder. Payment is for your work and does not depend on agreement with another coder or an expected answer. If you withdraw, your time worked is paid at the same rate within the agreed A$75 limit, and completed work is retained. These terms are identical for everyone and will be acknowledged privately before work starts. Stop at 90 minutes if unfinished and report that fact privately without discussing item content; do not undertake additional unpaid work. A reserve receives no work or packet, and incurs no commitment, unless activated to replace a primary coder.

This posting collects applications. Screening begins with the questions below and a review of platform evidence. Any later paid screening task will be agreed separately before it starts. Do not submit sample labels or undertake unpaid trial work with your bid. A start date will be agreed privately after screening.

Independence conditions
1. Complete the work personally and alone. Do not subcontract it or discuss the instructions, items or answers with anyone.
2. Do not use an LLM, chatbot, search engine, external reference or software assistant to interpret or label any item. Ordinary software used only to open and save the response file is permitted.
3. Use only your assigned packet. Do not look for another coder, their packet, an answer key or the researcher's intended results.
4. Questions about logistics or a file that will not open are welcome privately. Do not send item content or ask about its meaning, wording or category until the coordinator confirms that both independent returns have been frozen.
5. Keep all supplied materials private. If you discover relevant prior exposure or a possible connection to another coder, disclose it immediately. Disclosure will not reduce payment already earned under the agreed terms.

Prior exposure
You must not have encountered this research project's materials before this recruitment notice. Do not search for the project or the repositories mentioned below to prepare your answers. Answer from what you already know. An honest “not sure” requires clarification and is not an automatic rejection.

Required application questions
Please number your answers. Use Yes / No / Not sure plus one sentence for questions 1 and 2.
1. Have you ever read, commented on, reacted to, forked or submitted code to any public issue or pull request in lighteval, ragas, HELM, HuggingFace evaluate or deepeval about how those tools handle missing, unparseable or empty values?
2. Before seeing this recruitment notice, had you read anything written by or about this research project: a paper, preprint, funding page, forum post or code repository? If you cannot identify whether something you saw was this project, answer “Not sure”; do not investigate it.
3. How did you come across this posting? Name the route, such as Freelancer search or a referral. Do not publish someone else's private details.
4. Will you personally work alone, without subcontracting, discussion or assistance interpreting the items, and accept all five independence conditions above?
5. In two sentences, explain how you would follow written category rules when you personally disagree with them, and what you would do if a file would not open. Do not invent or label a sample exchange.
6. Can you complete the main task in one sitting, and what dates and time zone suit you? Confirm whether you are applying as a primary coder, reserve, or either, and whether the proposed rate and limit are acceptable.

Deliverable
One CSV with exactly the header coder_role,item_id,label and one row per item. Keep your assigned coder_role unchanged. Choose one of the six labels specified in your packet. Return no explanations, confidence ratings or extra columns in the main-task CSV. Return the file privately through Freelancer.

Applications will be reviewed after the bidding period shown on Freelancer ends. Communicate about logistics through private Freelancer messages. Do not post personal details, research materials or item answers in public comments. Only screened and separately appointed coders will receive their own packet after acknowledging its receipt.