Member of Technical Staff - Pre-Training Infra

Highlights

• Build and scale systems powering frontier model pre-training.
• Enable rapid experimentation on new training techniques at scale.
• Help define the future of open foundational models.

What they offer

• Top-tier compensation with salary and equity
• Stock options for all contributors
• Comprehensive medical, dental, vision, life and wellness allowance
• Lunch and dinner provided in the office daily
• 22 weeks paid parental leave for all new parents
• Unlimited PTO in the U.S. and 30 days in the U.K.
• Regular off-sites, happy hours and team celebrations

Ideal candidate

Systems-minded engineer who thrives alongside researchers, turning experimental ideas into stable, large-scale training. You care about performance, reliability, and making open intelligence accessible to all.

Requirements

• Experience building or operating distributed training systems for large machine learning models
• Strong experience with modern distributed training frameworks such as Megatron, DeepSpeed or similar
• Familiarity with large-scale model parallelism strategies (data, tensor, pipeline, or expert parallelism)
• Experience optimizing training throughput and GPU utilization in large distributed environments
• Familiarity with GPU communication libraries such as NCCL and performance tuning
• Experience working closely with ML researchers to productionize experimental training workflows
• Strong debugging skills across GPU compute, distributed training systems and large-scale ML pipelines
• Experience with large datasets and training pipelines for foundation model pre-training