Research Engineer, Web Crawling
Thinking Machines Lab · San Francisco · New York
Frontier labonsitesenior$350k–$475k
At a glance
Highlights
- Own distributed web crawlers that source pretraining data at internet scale.
- Build pipelines turning petabyte-scale raw crawls into usable training data.
- Partner with pretraining team on how crawled data affects model performance.
What they offer
- Generous health, dental, and vision benefits
- Unlimited PTO
- Paid parental leave
- Relocation support as needed
Ideal candidate
A seasoned builder of internet-scale data systems who loves owning production infrastructure end to end and wants crawled data to directly move frontier model quality.
Requirements
- 8+ years designing, building, and scaling web crawlers, scrapers, or large-scale distributed data-acquisition systems
- Track record of owning crawler or data-acquisition infrastructure at internet scale
- Strong software engineering skills with real experience in distributed systems
- Working knowledge of practical and legal considerations of large-scale web data collection
- visa sponsorship
- We sponsor visas
Lab profile
About Thinking Machines Lab
Builds frontier AI that extends human will and judgment through customizable models and interfaces.
What they build
- Inkling frontier models
- Tinker model customization
- Human-AI communication interfaces
Known for
- Frontier model training
- Post-training and reinforcement learning
- Evaluation infrastructure
- GPU supercomputing and reliability
Posted · Last seen