Russian Audio Generalist Evaluator Expert (San Francisco Bay Area)

Content & Media • Independent Contractor • Remote • Weekly Payouts

Mercor is seeking a Russian Audio Generalist Evaluator Expert to contribute to a high-impact audio AI research project with a leading research lab. In this role, you will work on transcription, annotation, and evaluation tasks that help train and benchmark advanced language models. This is a short-term, structured engagement ideal for candidates with strong academic or analytical backgrounds who are fluent in Russian and English and are based in the San Francisco Bay Area, enabling occasional in-person collaboration if required.

Job Responsibilities

Transcribe and Optimise Audio & Video

• Listen to, analyse, and transcribe audio and video content in Russian, following detailed constraints and instructions.

• Produce high-quality written outputs in Russian, with supporting work in English when required.

• Ensure clarity, accuracy, and strict adherence to formatting and stylistic guidelines.

• Capture nuances such as tone, intent, formal vs. informal register, regional expressions, dialectal variations, and contemporary Russian usage where relevant.

Define and Document Evaluation Standards

• Establish clear expectations for correct and high-quality responses in general consumer audio contexts.

• Develop detailed evaluation rubrics and grading guidelines in Russian and English.

• Document standards to ensure consistency across reviewers and model evaluations.

• Identify linguistic nuances, grammatical complexities, colloquialisms, and edge cases specific to Russian.

Conduct Model Testing and Grading

• Run prompts through language models and assess generated outputs.

• Evaluate responses against predefined criteria for accuracy, completeness, fluency, and instructional clarity.

• Provide structured feedback to improve model performance in Russian audio tasks.

Support Benchmarking and Quality Assurance

• Participate in QA and review cycles to ensure tasks, rubrics, and outputs meet Mercor’s quality bar.

• Maintain consistency and reliability before datasets are integrated into official benchmarks.

• Collaborate with project leads to resolve ambiguities and improve task design.

Minimum Qualifications

• Strong writing, editing, and critical thinking skills.

• Ability to work independently, manage time effectively, and meet deadlines.

• Native or near-native fluency in Russian (spoken and written) and professional fluency in English.

• Strong familiarity with spoken Russian, regional vocabulary, dialects, and contemporary language usage.

• Ability to accurately transcribe and analyse Russian audio content across general consumer contexts.

• Must be based in the San Francisco Bay Area.

• Available to commit 10, 20 hours per week.

Preferred Qualifications

• College students or recent graduates.

• Background in linguistics, humanities, social sciences, journalism, translation/localization, or technical disciplines.

• Prior experience with transcription, annotation, localisation, evaluation, or research workflows in Russian.

• Familiarity with regional variations of Russian and contemporary digital language usage.

• Interest in AI, language models, or applied research environments.

Application & Onboarding Process

• Complete a short AI-led interview (approximately 15 minutes).

• If selected, you will be onboarded and invited to begin project work.

Details

• Pay: $50 per hour (hourly)
• Commitment: 40 hours per week • remote
• Eligible locations: United States
• Platform: Mercor (weekly payouts via Stripe or Wise)

Submit your application via the link below. Qualified candidates move through Mercor's short AI interview and selection process. Projects can be extended, shortened, or concluded early depending on needs and performance.