Senior Machine Learning Engineer, LLM Inference Optimization
Nebius · Amsterdam, Netherlands · Berlin, Germany · Israel · Remote - Europe
Infrastructureremotesenior
At a glance
Highlights
- Own model and endpoint optimization from artifacts to production deployment.
- Improve latency, throughput and cost per token while preserving quality.
- Validate improvements via reproducible benchmarks and safe production rollouts.
What they offer
- Competitive compensation
- Career growth and learning opportunities
- Flexibility and ownership
- Collaborative and innovative culture
- Opportunity to work on impactful AI projects
- International environment and talented teams
Ideal candidate
A performance-obsessed ML engineer who loves digging into model internals and production systems to make inference faster and cheaper without sacrificing quality. You thrive collaborating with platform engineers and owning optimization end to end.
Requirements
- Strong engineering skills in Python and PyTorch
- Experience with distillation, speculative decoding, EAGLE, Medusa, multi-token prediction, or related inference acceleration methods
- Experience supporting agentic workloads involving tool calling, structured outputs, streaming APIs, high concurrency, and multi-step orchestration
- Familiarity with CUDA or Triton
- Contributions to open-source projects such as vLLM, SGLang, TensorRT-LLM, FlashInfer, LMCache, PyTorch, Triton, Ray, or KServe
- Implementation experience with cache-aware request routing, PDD, or WideEP, including diagnosing communication bottlenecks and load imbalance
- Familiarity with GPU communication libraries and interconnects, such as NCCL, NVLink, InfiniBand, or RoCE
Lab profile
About Nebius
Full-stack AI cloud platform for data, model training, and production deployment at scale.
What they build
- Full-stack AI cloud platform
Known for
- Large-scale GPU orchestration
- Inference optimization
- Efficient LLM inference and model optimization
- Physical AI infrastructure
Posted · Last seen