About the job
Why Join Nebius
Nebius is at the forefront of revolutionizing cloud computing to fuel the global AI economy. We empower our clients with the tools and resources necessary to tackle real-world challenges and revolutionize industries, all while minimizing infrastructure costs and eliminating the need for extensive in-house AI/ML teams. Our workforce operates at the cutting edge of AI cloud infrastructure, collaborating with some of the most innovative leaders and engineers in the industry.
Our Work Environment
With our headquarters in Amsterdam and being listed on Nasdaq, Nebius boasts a global presence with R&D hubs in Europe, North America, and Israel. Our team comprises over 1,400 employees, including more than 400 skilled engineers with profound expertise in hardware and software engineering, complemented by an in-house AI R&D team.
The Role
The Tavily team is developing a groundbreaking search engine specifically designed for AI agents. We envision a future where AI agents drive the workforce, necessitating a fundamental shift in web access. Our Search API enables AI agents to perform real-time web searches, optimized for Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG). Supported by prominent investors, we cater to developers and enterprises worldwide. Our team is dynamic and ambitious, focusing on rapid deployment, constant iteration, and a strong impact.
As a Software Engineer specializing in Scraping, you will construct systems that facilitate AI interaction with the live web. Your responsibilities will include designing and implementing large-scale, intelligent web acquisition pipelines, distilling the vast and ever-evolving online ecosystem into structured, high-quality data.
This position requires a blend of technical expertise in browsers, networks, and automation, along with practical engineering skills. You will develop adaptive systems capable of navigating contemporary websites, understanding evolving web interactions and detection systems, and gathering meaningful data safely, efficiently, and at scale.
Your contributions will directly influence Tavily’s capacity to equip AI agents with accurate, real-time insights about the world, effectively bridging the gap between the open web and intelligent reasoning.
Your Responsibilities
- Create distributed data acquisition systems to capture and structure the live web.
- Analyze browser internals, fingerprinting, and anti-automation systems to develop stealthy, adaptive orchestration layers.
- Prototype and implement intelligent automation frameworks using tools like Playwright, Puppeteer, and low-level browser control protocols (CDP).
- Conduct in-depth research into network flows, JavaScript protections, and emerging web standards.
