Staff Software Engineer (Agentic Search) - Crawler
Nebius · Zurich, Switzerland · Amsterdam · London · Remote - Europe · Spain · United Kingdom
Infrastructureremotestaff
At a glance
Highlights
- Build agent-native search APIs for AI agents to retrieve real-world data at scale.
- Design distributed crawlers processing billions of URLs with freshness focus.
- Own web-scale acquisition balancing coverage, efficiency and reliability.
What they offer
- Competitive compensation
- Career growth and learning opportunities
- Flexibility and ownership
- Collaborative and innovative culture
- Opportunity to work on impactful AI projects
- International environment and talented teams
Ideal candidate
Seasoned backend engineer who thrives on internet-scale distributed challenges and owns reliable production systems end-to-end. Passionate about grounding AI agents in fresh, verified web data through efficient crawling.
Requirements
- 8+ years of experience building backend or distributed systems
- Strong Go or C++ expertise
- Experience with large-scale distributed systems (10k+ RPS, billions of URLs, high-throughput pipelines)
- Understanding of web protocols (HTTP, DNS, TLS), crawling, scraping, and content extraction
- Experience operating production systems and debugging failures in distributed environments
- Strong understanding of scalability, fault tolerance, and resource management
- hiring process
- Coding interviews as part of the process
Lab profile
About Nebius
Full-stack AI cloud platform for data, model training, and production deployment at scale.
What they build
- Full-stack AI cloud platform
Known for
- Large-scale GPU orchestration
- Inference optimization
- Efficient LLM inference and model optimization
- Physical AI infrastructure
Posted · Last seen