Member of Technical Staff - Pre-Training Evals Infra

Highlights

• Build evaluation infrastructure guiding pre-training and release decisions.
• Create dashboards for comparing checkpoints and debugging behavior.
• Optimize inference throughput and GPU utilization for faster results.

What they offer

• Salary and equity with stock options for all employees
• Comprehensive medical, dental, vision, life insurance and wellness allowance
• Lunch and dinner provided in the office daily
• 22 weeks paid parental leave for all new parents
• Unlimited PTO in the U.S. and 30 days in the U.K.
• Visa sponsorship and long-term immigration support
• Regular off-sites, happy hours and team celebrations

Ideal candidate

A systems-minded engineer who loves partnering with researchers, owning ambiguous problems end-to-end, and sweating subtle discrepancies to make evals fast, reliable, and trustworthy.

Requirements

• Strong proficiency in Python, C++, or Rust with deep reasoning about concurrency and performance
• Experience designing, building, and operating reliable, maintainable distributed systems
• Working knowledge of model inference and evaluation and how implementation affects quality
• Ability to investigate complex cross-system problems and identify root causes
• Strong product judgment translating researcher needs into clear APIs and usable tools
• Comfort owning ambiguous problems end-to-end and balancing speed with reliability
• Attention to detail and persistence investigating subtle discrepancies in outputs and results