Training Performance Engineer
OpenAI · San Francisco
CHECKED 23H AGO
Frontier labremote$295k–$500k
At a glance
Highlights
- Drive efficiency gains across distributed training powering frontier-scale models.
- Profile end-to-end training runs to push throughput and uptime boundaries.
- Build observability to boost training and researcher throughput at scale.
What they offer
- Equity
- Relocation assistance
Ideal candidate
Systems-obsessed optimizer who loves digging through profiling data to squeeze every flop from large clusters and partnering with researchers to turn insights into faster, more reliable training.
Requirements
- Strong programming skills in Python and C++
- Experience running distributed training jobs on multi-GPU systems or HPC clusters
- Exposure to frameworks like PyTorch, JAX, or TensorFlow and large-scale training loops
- Experience debugging complex distributed systems and measuring efficiency rigorously
- Comfortable collaborating across teams and translating profiling data into improvements
- Familiarity with NCCL, MPI, or UCX communication libraries
- Experience with large-scale data loading and checkpointing systems
- remote policy
- Hybrid, 3 days in office per week
Lab profile
About OpenAI
Builds safe frontier AI models and products like ChatGPT for broad benefit.
What they build
- ChatGPT
- OpenAI API
- Frontier AI models
Known for
- Frontier-scale distributed ML training
- Massive GPU fleet and data center operations
- Exabyte-scale data lakes and streaming
- AI alignment and safety research
Key facts
- Operates one of the world's largest GPU fleets