Software Engineer, LLM Performance & Evaluation

Highlights

• Own LLM inference performance and accuracy tracking on Furiosa NPUs.
• Build benchmarks isolating batching, caching and speculative decoding impact.
• Maintain dashboards tracking performance by model, hardware and software version.

Ideal candidate

A rigorous, curious performance detective who loves making complex trade-offs visible and guiding engineering decisions with reproducible evidence.

Requirements

• Strong Python programming skills and experience building maintainable automation, data pipelines, or engineering tools.
• Hands-on experience measuring and analyzing ML inference or systems performance, including profiling and latency/throughput analysis.
• Experience benchmarking GPU, NPU, or TPU systems and analyzing memory bandwidth, kernel execution, or communication bottlenecks.
• Experience operating automated evaluation workloads in Linux, container, or cluster environments with CI and dashboard integration.
• Ability to read and instrument Rust or C++ code.