AI Content Evaluation Specialist

I need an experienced evaluator who can look at ChatGPT-style text responses and judge them for factual accuracy, clarity, tone, and overall usefulness. You will also design the prompts and rubrics that drive those evaluations, then summarise your findings so our engineering team can refine the model.

The day-to-day work is fully remote and largely self-directed, but you’ll still collaborate with linguists, data scientists, and product managers when guidelines change or new use-cases appear. Clear, concise writing, sharp critical thinking, and a knack for spotting subtle errors are essential. Backgrounds in data annotation, research, teaching, technical writing, or previous AI evaluation projects tend to translate well here, though they are not strict requirements.

Deliverables I expect from each assignment:
• A scored evaluation sheet for every batch of model outputs you review
• A short narrative report explaining recurrent issues and proposed fixes
• A set of revised or new prompts designed to elicit higher-quality answers

Work quality will be checked against our internal rubric for consistency, completeness, and the practicality of your recommendations. If that sounds like a good fit, let’s talk about the first evaluation batch and timeline.