Bilingual German Evaluator

Mercor connects elite creative and technical talent with leading AI research labs. The company is headquartered in San Francisco and has notable investors including Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.

Location: Remote

Salary: $25–$30/hour

Responsibilities

• Design detailed prompts in German and/or English with constraints and instructions for real-world relevance in Austria, Switzerland, and Germany.
• Define evaluation standards and develop rubrics considering linguistic nuance and cultural conventions in these regions.
• Test models by running prompts and assess outputs for accuracy, fluency, and cultural fit in German and English.
• Collaborate in quality assurance to ensure consistency and reliability across German-language benchmarks.
• Work independently and meet deadlines while contributing to a structured project environment.

Requirements

• Native-level fluency in German specific to Austria, Switzerland, or Germany.
• Must be native to Austria, Switzerland, or Germany with deep cultural familiarity.
• BS or BA from a reputable institution.
• Strong writing and critical thinking skills.
• Familiarity with ChatGPT or similar tools.
• Based in Austria, Switzerland, or Germany.