Senior AI R&D Engineer - Performance - Pre-training (f/m/d)

Highlights

• Maximize training throughput on clusters with thousands of GPUs.
• Engineer systems to train European foundation models like Kolibri-1 at scale.
• Shape hardware-efficient model architectures with AI researchers.

What they offer

• 30 days of paid vacation
• Access to fitness & wellness offerings via Wellhub
• Mental health support through nilo.health
• Substantially subsidized company pension plan
• Subsidized Germany-wide transportation ticket
• Budget for additional technical equipment
• Flexible working hours
• Hybrid working model
• JobRad Bike Lease

Ideal candidate

A performance-driven systems thinker who thrives on ownership and optimizing large-scale training, excited to collaborate on-site in Heidelberg to make fast, reliable European foundation models.

Requirements

• Proficient in Python and the PyTorch library
• Strong engineering background in parallel and/or distributed systems with proven track record of excellence
• Hands-on experience with modern machine learning techniques, especially large language models and their life cycle
• Deep understanding of the CUDA programming model
• Experience in distributed programming with APIs like NCCL or MPI
• Experience analysing profiling traces with tools such as PyTorch Profiler and Nvidia Nsight
• Contributions to modern distributed training frameworks (e.g., TorchTitan, Megatron-LM, DeepSpeed)
• Familiarity with low-precision training formats (MXFP4, MXFP8) and their impact on numerical stability and throughput
• Deep understanding of NCCL communication primitives, NVSHMEM or CUDA IPC and their performance
• Proven track record of implementing and optimising modern transformer-based model training
• Proven track record working on the NVIDIA Blackwell architecture