Inference Engineer
General Compute · San Francisco · NYC
Infrastructureremotesenior
At a glance
Highlights
- Own the serving layer that turns verified models into live customer traffic.
- Prove a hardware-specific serving stack delivers 5-7x faster token generation.
- Design autoscaling and failover for an ASIC fleet under real customer load.
Ideal candidate
Self-directed infrastructure owner who thrives on-call and designs for failure, eager to own reliability end-to-end and define a new serving stack from scratch at early stage.
Requirements
- 5+ years building and operating production systems at the infrastructure layer, ideally high-throughput or low-latency serving
- Direct experience with LLM inference serving including request batching, KV-cache management, or continuous batching in production
- Comfortable owning reliability and on-call for critical systems, designing for failure
- Strong systems fundamentals in concurrency, networking, and scheduling to reason about hardware-level performance
- Self-directed and comfortable with ambiguity as a founding role
Lab profile
About General Compute
Neocloud for alternative chips, running fragmented inference on purpose-built silicon.
What they build
- Alternative-chip inference cloud
- Split prefill-decode inference serving
Known for
- Productionizing SambaNova, Cerebras, Positron, d-Matrix hardware
- Rack procurement and data center operations
- ML compiler and model bring-up
Posted · Last seen