Research Engineer - Web Crawlers

Highlights

• Own crawling systems sourcing world-class data for frontier AI models.
• Build pipelines turning audio, video, multilingual content into training datasets.
• Create tooling letting researchers request and explore newly crawled data fast.

What they offer

• Annual discretionary learning & development stipend
• Annual discretionary social travel stipend to meet colleagues
• Annual company offsite bringing entire team together
• Monthly co-working stipend if not near main hubs

Ideal candidate

Autonomous, impact-driven builder fascinated by web-scale data quality who thrives owning complex systems end-to-end and turning messy open-web content into frontier-model fuel.

Requirements

• Hands-on experience building and scaling web crawlers or scraping systems, ideally for ML training data
• Strong engineering skills in distributed systems at scale processing billions of documents
• Ability to autonomously evaluate quality, coverage and compliance of crawled data and build measurement tooling