Research Engineer, Intern – Benchmarking

Highlights

• Own benchmarking pipelines measuring tool use and agentic behavior.
• Build eval systems, scoring and dashboards for frontier models.
• Turn failure analysis into improvements for post-training and data pipelines.

What they offer

• Health benefits eligible at 30+ hours a week
• Overtime eligible at 1.5x from 40 to 55 hours; 2x over 55 hours
• $1.5K monthly stipend for meals
• Relocation & housing support
• Free Equinox membership
• $200 monthly laundry reimbursement
• $200 monthly personal wellness reimbursement

Ideal candidate

An applied researcher energized by dissecting model behavior and building rigorous evals, thriving with high ownership in a fast-paced, in-person research environment.

Requirements

• Strong applied research background, with focus on model evaluation, benchmarking, and/or failure analysis
• Strong coding skills and hands-on experience with ML models and evaluation code
• Solid grasp of data structures, algorithms, and backend systems
• Comfort with APIs, SQL/NoSQL, and cloud platforms for running and storing eval results
• Ability to reason about model behavior, experimental results, and data quality from evals and failure analyses
• Excitement to work in person in San Francisco five days a week in a high-intensity, high-ownership environment