Backend Engineer

Highlights

• Own Jan Router, an enterprise LLM gateway, from scratch to production.
• Build billing-grade routing, metering and org controls for LLM traffic.
• Wire Jan Agent's model and tool calls through the same gateway.

Ideal candidate

A pragmatic, ownership-driven backend builder who ships zero-to-production systems and thrives in ambiguity. You work agent-natively, weigh speed versus correctness, and care about open, private AI built in public.

Requirements

• Work agent-natively by default, running multiple agents in parallel with review discipline
• Acquainted with LLM API systems: providers, OpenAI-compatible endpoints, streaming, tool calling, token accounting
• Experience with LLM infra: inference proxies, provider SDKs, token counting, or gateways like LiteLLM
• Acquainted with core AI concepts: context windows, inference, prompting, evals, open vs hosted models
• Proven ability to ship from zero to production and own infra, deploy, alerting, and CI/CD
• Open-minded and pragmatic, comfortable with full ownership and ambiguity
• Track record building production backend services that handle real traffic
• Solid datastore skills: PostgreSQL, Redis, queues, and schema design for usage and billing data
• Strong in at least one of TypeScript or Python
• Comfort with Docker, Kubernetes, OAuth and OIDC, API keys, RBAC, tenant isolation, secure secrets handling
• Familiarity with the Jan.ai ecosystem or other OSS LLM tooling
• MCP and tool-calling knowledge for agent and tool traffic