Senior Research Scientist, Model Evaluation

Highlights

• Create next-generation evaluation methods to measure frontier LLM progress.
• Build ambitious new benchmarks that push limits of model capabilities.
• Advance state-of-the-art LLM evaluation including training LLM judges.

What they offer

• Weekly lunch stipend
• Full health and dental benefits including mental health budget
• RRSP matching, 401K, Pension Scheme
• 100% parental leave top-up for up to 6 months
• 6 weeks of paid vacation
• $500 home office stipend
• Annual enrichment, education and co-working benefits

Ideal candidate

A research-minded builder obsessed with rigorously measuring AI capabilities and aligning metrics with real behavior. You love prototyping to probe LLM limits and digging through complex outputs to ensure high quality.

Requirements

• Enjoy rapidly building prototypes that demonstrate boundaries of LLMs and developing resources to measure those capabilities
• Spent dozens of hours reviewing complex data and LLM outputs to ensure high data quality
• Obsessive about rigorously measuring AI capabilities and aligning measurements with capabilities you care about
• Strong software engineering skills