Member of Technical Staff, Data Analysis and Evaluation

Highlights

• Ensure quality and reliability of frontier large language models for enterprise.
• Design data collection tasks and evaluate dataset quality and robustness.
• Analyze model generalizability across diverse real-world business scenarios.

What they offer

• Weekly lunch stipend
• Full health and dental benefits including mental health budget
• RRSP matching, 401K, Pension Scheme
• 100% parental leave top-up for up to 6 months
• Annual enrichment, education and learning benefits
• 6 weeks of paid vacation
• Travel budget for remote office visits plus annual offsite
• $500 home office stipend and co-working benefit

Ideal candidate

A statistically-minded ML practitioner passionate about data quality and rigorous evaluation who thrives collaborating with researchers and annotators to make frontier models robust and impactful.

Requirements

• Extremely strong software engineering skills
• Strong expertise designing and conducting data collection tasks including working with human annotators
• Strong statistical skills and experience evaluating scientific experiments
• Experience analysing datasets for quality, biases and suitability for training ML models
• Hands-on experience training LLMs on distributed training infrastructures
• Familiarity with evaluating and improving generalisability and robustness of ML systems
• Proficiency in Python and ML frameworks such as PyTorch, TensorFlow, JAX
• Excellent communication skills for cross-functional collaboration
• One or more papers at top-tier venues such as NeurIPS, ICML, ICLR, ACL, EMNLP