Long-term Python Expert for AI Training
Budget / SalaryHourly project
TypeFreelance project
LocationRemote
Posted2 hours ago
AI Training & Evaluation Specialist — Python / Data Science | Long-Term Contract (2+ years)
Summary
We’re looking for an experienced AI training and evaluation specialist for a long-term freelance collaboration supporting authorized projects across platforms and vendors such as DataAnnotation.tech, micro1, Mercor, Snorkel AI, Handshake, and similar services.
We need someone who can do more than basic data labeling. The ideal candidate can design challenging data-science tasks, create high-quality datasets, evaluate LLM responses, and develop clear scoring rubrics.
Our goal is to build an ongoing working relationship with a reliable expert—not hire someone for a single task. Work volume will depend on project availability, client approval, and performance.
Responsibilities
- Design original data-science and coding tasks that expose meaningful errors in advanced AI models.
- Create and validate synthetic datasets that meet project requirements.
- Test GPT, Claude, and other models through approved APIs and evaluation tools.
- Develop reference solutions, evaluation rubrics, and supporting documentation.
- Identify issues such as data leakage, flawed statistical reasoning, incorrect assumptions, and coding errors.
- Review AI-generated responses for correctness, completeness, and reproducibility.
- Follow detailed instructions and maintain accurate records of work performed.
- Communicate progress, blockers, and quality concerns proactively.
Required Qualifications
- Strong Python skills, including pandas, NumPy, and scikit-learn.
- Solid understanding of statistics, machine learning, and practical data science.
- Hands-on experience with LLM evaluation, AI training, technical annotation, or model red-teaming.
- Experience working with APIs and debugging technical workflows.
- Excellent written English and attention to detail.
- Ability to work independently and follow strict quality standards.
Preferred Qualifications
- Previous experience with DataAnnotation, micro1, Mercor, Snorkel AI, Handshake, or comparable vendors.
- Experience building evaluation rubrics, benchmark tasks, or reference answers.
- Familiarity with Codex or similar AI-assisted coding tools.
- Experience generating and validating synthetic datasets.
Engagement Details
- Contract type: Long-term freelance engagement.
- Initial stage: Paid trial using an original or otherwise authorized evaluation task.
- Ongoing work: Based on trial results, project availability, and applicable client permissions.
- Hours: Please include your weekly availability.
- Compensation: $2~$8/hr
How to Apply
Please include:
1. A summary of your relevant AI training and evaluation experience.
2. The platforms or vendors you have worked with
3. Your Python, statistics, and machine-learning background.
4. One example of a challenging model-evaluation task you designed or reviewed.
5. Your hourly rate, weekly availability, time zone, and earliest start date.
6. Whether you are applying as an individual or an agency.
We’re especially interested in candidates seeking a stable, long-term collaboration who consistently deliver thoughtful, accurate technical work.
Summary
We’re looking for an experienced AI training and evaluation specialist for a long-term freelance collaboration supporting authorized projects across platforms and vendors such as DataAnnotation.tech, micro1, Mercor, Snorkel AI, Handshake, and similar services.
We need someone who can do more than basic data labeling. The ideal candidate can design challenging data-science tasks, create high-quality datasets, evaluate LLM responses, and develop clear scoring rubrics.
Our goal is to build an ongoing working relationship with a reliable expert—not hire someone for a single task. Work volume will depend on project availability, client approval, and performance.
Responsibilities
- Design original data-science and coding tasks that expose meaningful errors in advanced AI models.
- Create and validate synthetic datasets that meet project requirements.
- Test GPT, Claude, and other models through approved APIs and evaluation tools.
- Develop reference solutions, evaluation rubrics, and supporting documentation.
- Identify issues such as data leakage, flawed statistical reasoning, incorrect assumptions, and coding errors.
- Review AI-generated responses for correctness, completeness, and reproducibility.
- Follow detailed instructions and maintain accurate records of work performed.
- Communicate progress, blockers, and quality concerns proactively.
Required Qualifications
- Strong Python skills, including pandas, NumPy, and scikit-learn.
- Solid understanding of statistics, machine learning, and practical data science.
- Hands-on experience with LLM evaluation, AI training, technical annotation, or model red-teaming.
- Experience working with APIs and debugging technical workflows.
- Excellent written English and attention to detail.
- Ability to work independently and follow strict quality standards.
Preferred Qualifications
- Previous experience with DataAnnotation, micro1, Mercor, Snorkel AI, Handshake, or comparable vendors.
- Experience building evaluation rubrics, benchmark tasks, or reference answers.
- Familiarity with Codex or similar AI-assisted coding tools.
- Experience generating and validating synthetic datasets.
Engagement Details
- Contract type: Long-term freelance engagement.
- Initial stage: Paid trial using an original or otherwise authorized evaluation task.
- Ongoing work: Based on trial results, project availability, and applicable client permissions.
- Hours: Please include your weekly availability.
- Compensation: $2~$8/hr
How to Apply
Please include:
1. A summary of your relevant AI training and evaluation experience.
2. The platforms or vendors you have worked with
3. Your Python, statistics, and machine-learning background.
4. One example of a challenging model-evaluation task you designed or reviewed.
5. Your hourly rate, weekly availability, time zone, and earliest start date.
6. Whether you are applying as an individual or an agency.
We’re especially interested in candidates seeking a stable, long-term collaboration who consistently deliver thoughtful, accurate technical work.
Apply on Freelancer →
Project sourced from Freelancer.com. Applications happen directly on the original platform — we never collect your data.