Freelance Agent Evaluation Engineer

Building a dataset to evaluate AI coding agents by creating challenging tasks and evaluation criteria within realistic simulated environments. The role involves creating tasks for AI agents to evaluate and improve their coding abilities on a part-time, non-permanent basis.

Location: Remote

Salary: up to $45 per hour

Responsibilities

• Create tasks for AI agents to evaluate and improve their coding abilities.

Requirements

• Degree in Computer Science, Software Engineering, or related fields
• 5+ years in software development, primarily Python
• Background in full-stack development, with experience building React-based interfaces and robust back-end systems
• Experience writing tests, familiarity with Docker containers, CI/CD tools, and infrastructure tools

Benefits

• Opportunity to work on a challenging project
• Flexible schedule
• Compensation up to $45 per hour