Qualifications
Key ResponsibilitiesDesign and implement high-performance, scalable data pipelines for various AI and machine learning projects. Architect multi-region data infrastructure to ensure global data availability and synchronization. Develop flexible pipeline architectures for complex branching and logic isolation to support multiple concurrent AI initiatives. Optimize large-scale data processing workflows using Databricks and Spark to enhance throughput and reduce costs. Maintain and improve Kubernetes-based containerized data environments for reliable data workload execution. Collaborate with AI research and platform teams to ensure a seamless supply of high-quality data into training and evaluation pipelines.
About the job
Join Our Dynamic Team
At 42dot, we are on the lookout for a talented Senior AI Data Pipeline Engineer to design, develop, and enhance our global data pipelines. You will play a crucial role in managing and processing data collected from around the world, ensuring that our high-throughput systems reliably deliver petabyte-scale data to our vast GPU infrastructure, thereby powering essential AI workloads.
As part of our team, you will have the opportunity to architect scalable solutions that support diverse AI and machine learning projects, while collaborating closely with AI researchers and platform teams to ensure efficient data flow into training and evaluation pipelines.
About 42dot
42dot is at the forefront of AI and data engineering, dedicated to developing cutting-edge solutions that drive innovation and efficiency in data management. Our team thrives on collaboration and continuous improvement, making us an exciting place to grow your career.
