AI Response Rater Training
Budget / SalaryHourly project
TypeFreelance project
LocationRemote
Posted54 minutes ago
I need a concise, practitioner-ready training package that teaches new evaluators how to rate AI-generated text responses. The goal is to give them clear, consistent guidelines so every judgment they make is aligned on safety, fairness, and respectful language.
Here’s what I’m looking for:
• A step-by-step guide that explains the rating framework, the meaning of each scale point, and why safety, fairness, and respect matter.
• At least 20 sample AI answers covering a variety of real-world prompts, each fully annotated to show model strengths and weaknesses in those three areas.
• A short quiz or calibration exercise my team can take after reading the material; it should include an answer key and rationale so we can self-calibrate.
• Suggestions on how to expand the rubric later if we decide to add accuracy or instruction-following as additional dimensions.
Plain-language writing, compact formatting, and easy copy-paste tables are all appreciated so I can drop the content straight into my internal onboarding docs. I’d like the first draft within a week, with two quick revision passes if needed to tighten wording or adjust examples.
Here’s what I’m looking for:
• A step-by-step guide that explains the rating framework, the meaning of each scale point, and why safety, fairness, and respect matter.
• At least 20 sample AI answers covering a variety of real-world prompts, each fully annotated to show model strengths and weaknesses in those three areas.
• A short quiz or calibration exercise my team can take after reading the material; it should include an answer key and rationale so we can self-calibrate.
• Suggestions on how to expand the rubric later if we decide to add accuracy or instruction-following as additional dimensions.
Plain-language writing, compact formatting, and easy copy-paste tables are all appreciated so I can drop the content straight into my internal onboarding docs. I’d like the first draft within a week, with two quick revision passes if needed to tighten wording or adjust examples.
Apply on Freelancer →
Project sourced from Freelancer.com. Applications happen directly on the original platform — we never collect your data.