Applied Research Engineer, Evaluation
Black Forest Labs · Freiburg (Germany)
Frontier labonsite$168k–$246k
At a glance
Highlights
- Build the evaluation stack every FLUX checkpoint must pass to prove itself.
- Benchmark FLUX against frontier rivals to pinpoint real strengths and gaps.
- Design new ways to measure image, video, audio and action for FLUX 3.
What they offer
- Equity
- Reasonable travel costs covered
Ideal candidate
You’ve evaluated generative models hands-on and care about what outputs actually look like beyond aggregate scores. Statistically rigorous, low-ego and obsessed with getting measurement right to guide research.
Requirements
- Advanced degree (MS/PhD) in relevant field or equivalent practical experience
- Hands-on experience evaluating generative models
- Very strong Python and software engineering skills
- Working knowledge of PyTorch and modern ML infrastructure
- Strong grasp of statistics and its shortcomings in practice
- Experience operating LLM-as-judge systems and understanding limitations
- Working knowledge of modern benchmarking practices and standard datasets
- Inclination to inspect actual outputs beyond aggregate scores
- remote policy
- Hybrid, at least 2 days a week in Freiburg or one full week every other week
Lab profile
About Black Forest Labs
Team behind Stable Diffusion and FLUX building frontier image and video generation models.
What they build
- FLUX image generation models
- FLUX video generation models
Known for
- Pioneers of Latent Diffusion and Stable Diffusion
- State-of-the-art image and video generation
- Models used by millions of creators and businesses
Key facts
- Headquartered in Freiburg, Germany
- Presence in San Francisco
Posted · Last seen