Senior Staff LLM Inference Engineer
d-Matrix · Santa Clara
Infrastructureremotestaff$195k–$285k
At a glance
Highlights
- Build PoC systems showcasing in-memory compute for frontier LLM inference.
- Optimize kernels and distributed serving across heterogeneous hardware.
- Shape next-gen AI silicon roadmap working directly with hardware teams.
What they offer
- equity
- bonus
- benefits
Ideal candidate
End-to-end inference engineer with 10+ years experience who thrives on hard problems and owns ideas from prototype to deployment. Energized by novel hardware and motivated by shaping frontier AI infrastructure in a small, high-ownership team.
Requirements
- Bachelor's in Computer Science, Electrical Engineering or related field with 10+ years relevant experience
- Master's or PhD preferred with 6+ years relevant industry experience
- Deep intuition for generative AI architectures and inference performance optimization
- Familiar with internals of open-source inference frameworks such as vLLM, SGLang, TensorRT-LLM
- Experience with distributed inference including tensor/pipeline parallelism and disaggregated prefill/decode
Lab profile
About d-Matrix
Pioneers memory-centric compute-in-memory chips and software for faster, energy-efficient AI inference.
What they build
- Digital in-memory compute inference chips
- Rack-scale AI inference systems
- AI inference software stack
Known for
- Compute-in-memory architecture
- Energy-efficient inference design
- Full-stack chips-to-systems expertise
Key facts
- Founded in 2019
- Headquartered in Santa Clara, California
- Backed by Temasek, Playground Global, and M12
Posted · Last seen