Engineering Data Scientist - Evaluations
Sarvam · Bengaluru
CHECKED 16H AGO
Frontier labonsitemid-level
At a glance
Highlights
- Build domain-specific eval harnesses for high-stakes AI use cases.
- Design eval frameworks for document, geospatial and workflow AI.
- Own deployment readiness by surfacing model quality in production.
Ideal candidate
Skeptical, rigorous data scientist excited to own quality gates for high-stakes AI and hunt failure modes before deployment.
Requirements
- 3-6 years in data science, ML research, or applied AI with at least 2 years working with LLMs in production
- Strong statistics and probability fundamentals
- Experience designing evaluation frameworks from scratch: custom metrics, inter-rater reliability, red-teaming
- Python proficiency with pandas, NumPy, HuggingFace datasets, RAGAS, Eval Harness, LangSmith or equivalent
- Experience with prompt engineering, model fine-tuning, or RLHF in applied settings
- Ability to work with unstructured domain data: PDFs, doctrine documents, transcripts, and field reports
Lab profile
About Sarvam
Building India's full-stack sovereign AI platform across research, models, infrastructure and applications.
What they build
- Model API
- On-device AI
- Enterprise AI agents
Known for
- Sovereign AI focus for India
- Full-stack across research to applications
- Production deployment at banks, insurers and government
- Indian-language interactions at scale
Key facts
- Backed by Lightspeed, Peak XV and Khosla Ventures