Senior Backend Engineer, Inference Platform
You.com · San Francisco, CA, New York, NY (Remote) · United States - Remote
Infrastructureremotesenior$200k–$250k
At a glance
Highlights
- Own the inference gateway that handles every model call across products.
- Build intelligent routing that balances cost, latency and availability.
- Ship gateway changes in shadow mode with measurement to prove impact.
Ideal candidate
A backend builder who thrives on critical-path ownership and loves optimizing real-world reliability, cost, and latency. You care about graceful degradation and proving ideas with data, and you grow fast into unfamiliar areas.
Requirements
- Hands-on production experience with model serving and understanding of batching, KV/prefix caching, and latency tradeoffs
- Experience tuning a serving stack against live traffic and improving p99 latency in practice
- Fluency across provider APIs including tokenization, prompt-caching semantics, and usage reporting
- Experience building or operating a proxy, gateway, or data plane on the critical path of production traffic
- Comfort with streaming responses, partial state, mid-stream failures, and post-stream work
- Strong backend generalist background with willingness to learn new tools quickly
- remote policy
- Remote within the United States; preference for San Francisco Bay Area or New York
Lab profile
About You.com
Building AI search infrastructure for real-time, accurate, citation-backed retrieval for agents and enterprises.
What they build
- AI Search Infrastructure platform
- Proprietary vertical indexes
- LLM-optimized retrieval for agents and enterprises
Known for
- Real-time accurate retrieval
- Citation-backed trusted knowledge layer
- Search and LLM-optimized systems
Posted · Last seen