Model Evaluator, AI/ML
About this role
About Turing
Turing is one of the world’s fastest-growing AI companies, accelerating the advancement and deployment of powerful AI systems. You’ll be at the forefront of evaluating how LLMs interact with real code, influencing the future of AI-assisted software development. This is a unique opportunity to blend practical software engineering with AI research.
About the Role:
We are seeking experienced Software Engineers to support a frontier AI training and evaluation initiative focused on improving the coding capabilities of AI agents.
In this role, you will review and compare AI-generated coding solutions across real-world software engineering tasks. You will analyze complete agent execution workflows, evaluate code correctness, efficiency, safety, and overall quality, and provide structured ratings and evidence-based feedback to help improve AI-assisted software development.
What does day-to-day look like:
- You'll review three AI coding agents that were each given the same real developer request.
- You read each agent's full run: the commands it ran, the files it changed, what it checked and its final message.
- Then you judge which run did the job best.
For every task you
- Rate each run on 10 quality areas, such as following instructions, whether the code works, efficiency, safety, honesty and communication.
- Give each run an overall score from 1 to 5 and rank the three.
- Write a short rationale explaining your ranking, with evidence from the runs.
Required Skills: - Minimum 3+ years of overall experience
- Strong experience with at least one of the following languages: Python, JavaScript/TypeScript, Java, Go, PHP or similar.
- Proficiency with Shell commands, Git, Docker, and basic software pipeline setup.
- Comfortable running, modifying, and testing real-world projects locally.
Assessment & Selection Process
- Candidates must complete three practice evaluation tasks as part of the selection process.
- Each assessment requires quality ratings, overall scores, comparative rankings, and written rationales.
- A reviewer will assess each task for technical accuracy, consistency, and quality of reasoning.
- Candidates will be onboarded only after all three practice tasks have been approved.
IMPORTANT, STRICT NO-AI POLICY
- DO NOT USE CHATGPT OR ANY OTHER AI TOOL TO WRITE YOUR EVALUATION RATIONALES, EXPLANATIONS, OR REASONS.
- ALL RATIONALES AND REASONING MUST BE WRITTEN INDEPENDENTLY BY THE CANDIDATE.
- COMPLIANCE WILL BE CHECKED. ANY USE OF AI-GENERATED RATIONALES OR REASONS WILL RESULT IN AUTOMATIC REJECTION.
Perks of Freelancing With Turing
- Work in a fully remote environment.
- Opportunity to work on cutting-edge AI projects with leading LLM companies.
Offer Details
- Commitments Required: At least 8 hours per day and 40 hours per week with overlap of 4 hours with PST.
- Employment type : Contractor assignment (no medical/paid leave)
- Duration of contrac : 1 month; [expected start date is next week]
- Payment model: $25 per approved task
- Location: India, Pakistan, Nigeria, Kenya, Egypt, Ghana, Bangladesh, Turkey, Mexico, LA
Evaluation Process (approximately 75 mins) :
- Assessment based review
Key skills
Details
- Pay: Shared during the application process
- Company: Turing
- Location: Remote
- Eligible locations: India, Pakistan, Nigeria, Kenya, Egypt, Ghana and 3 more
- Platform: Turing
Frequently asked questions
How much does the Model Evaluator, AI/ML role pay?
Turing does not publish a pay rate for this Model Evaluator, AI/ML position. Pay is shared during the application process and depends on your experience and the scope of the work.
Is the Model Evaluator, AI/ML role remote, and can I work from home?
Yes. Turing lists this Model Evaluator, AI/ML position as fully remote, so you can work from home or anywhere with a reliable computer and internet connection.
Is Model Evaluator, AI/ML full time or part time?
Turing lists this as a contractor remote role. Full details are in the role description above.
Which countries can apply for Model Evaluator, AI/ML?
Turing lists eligibility for India, Pakistan, Nigeria, Kenya, Egypt, Ghana, Bangladesh, Turkey, and 1 more countries. Check the Turing listing for current requirements before applying.
Do I need experience to apply for Model Evaluator, AI/ML?
The requirements are listed in the role description above. Turing screens applicants through its own process, and relevant skills or credentials strengthen your application.
How do I apply for the Model Evaluator, AI/ML role, and is it legit?
Use the apply button on this page to go straight to the official Turing listing. Turing is a real hiring marketplace and applying is free. No legitimate employer charges you to apply.
Stay Updated on Roles Like This
Subscribe to receive fresh openings aligned with Finance & Accounting and AI training roles across Turing, Mercor and JobHub by NeonLabs