Data Infrastructure Software Engineer

Lead Python Engineer - Data Infrastructure About AscentAI AscentAI is at the forefront of developing intelligent software solutions tailored for risk and compliance teams within financial institutions. Our innovative platform simplifies complex regulatory information into actionable insights, empowering teams to mitigate risks, enhance operational efficiency…

About ZaimlerIn a world where AI agents struggle to reason over fragmented data, Zaimler emerges as the solution. Our mission is to unify disparate enterprise data across countless systems, providing a shared context, meaning, and structure. This transformation is essential as we transition from traditional copilots to fully autonomous agents, necessitating a new infrastructure layer that we are dedicated to building.At Zaimler, we are pioneering context infrastructure for the agentic era—a platform that autonomously discovers domain knowledge, maps intricate relationships, and equips AI agents with the semantic understanding required for precise and scalable operations. Envision knowledge graphs that facilitate real-time inference, tailored for systems that need to reason rather than merely retrieve data.Founded by industry veterans Biswajit Das (former VP Engineering at Truera and Chief Architect at Visa) and Sofus Macskassy (ex-Director of Engineering at LinkedIn), who notably built one of the largest knowledge graphs in production, Zaimler is a small, senior team at the seed stage, collaborating with major enterprises in sectors like insurance, travel, and technology. If you are passionate about creating the infrastructure that will support the next decade of AI advancements, we are eager to connect with you.The RoleWe are in search of a talented Data Infrastructure Engineer to establish the foundational distributed data layer that will power our semantic platform. In this role, you will be responsible for designing, building, and scaling systems that enable high-throughput data ingestion, transformation, and real-time processing.

Join arch.co as a Lead Infrastructure Engineer, where you will play a pivotal role in designing, implementing, and maintaining our cutting-edge infrastructure systems. Collaborate with cross-functional teams to ensure robust architecture and performance optimization, while mentoring junior engineers and providing technical leadership.

Stage is looking for an Infrastructure Data Engineer to join the team in Boston, MA. This position centers on building and maintaining the systems that move and organize data, making analytics and business intelligence possible across the company. Role overview The Infrastructure Data Engineer will design, implement, and support scalable data infrastructure. The work ensures that business needs are met as the company grows, and that analytics teams have reliable data to inform decisions. What you will do Design and implement data infrastructure to meet evolving business requirements Build and maintain data pipelines used for analytics and reporting Support business intelligence by ensuring data systems are reliable and accessible Requirements Stage is seeking candidates who have a strong interest in data engineering and are motivated to make a real impact on team and company goals.

Join our innovative team at alljoined as a Data Infrastructure Engineer where you will play a pivotal role in shaping our data architecture and ensuring the reliability and efficiency of our data systems. You will collaborate with cross-functional teams to design and implement scalable data solutions that empower data-driven decision-making.

Are you passionate about Python and eager to share your expertise with a vast community of developers?The Real Python tutorial team is renowned for delivering top-tier Python tutorials online. Our mission is to empower Python developers globally to enhance their skills.With over 3 million monthly visitors, we are excited about our journey thus far, but believe we can reach even greater heights!To elevate our tutorial quality and broaden our content offerings, we are seeking enthusiastic video course instructors who:Have a deep love for Python and a desire to assist learners in advancing their skillsUnderstand the significance of clarity and tone in educational video contentAspire to refine their craft and leverage our comprehensive publishing processCan commit to producing one or more new recordings each month and adhere to deadlinesThis position is fully remote. For more details, visit: realpython.com/jobs/video-course-instructorIdeal candidates will:Possess several years of programming experienceBe passionate about teaching “programming concepts” and have experience recording screencasts. The content you create will primarily be derived from existing written tutorials, so your ability to transform written material into engaging short videos is crucial.Have the ability to integrate Real Python into their weekly routine, as this role requires a notable time commitment.Joining the Real Python team comes with numerous benefits:Continuous Learning: Engage in ongoing learning and enjoy the process, enhancing your skills as a developer, writer, and communicator while forming valuable connections.Wide Reach: Our website attracts significant traffic—over 3 million visitors monthly and consistently growing. We are frequently highlighted in various Python publications and manage one of the largest email newsletters and social media channels in the community. Our YouTube channel boasts over 150,000 subscribers, ensuring that your published video series garners substantial viewership and appreciation.Content Enhancement: Upon submission of a video series to Real Python, our dedicated team will work with you to ensure the highest quality output.

Join our dynamic team at Sonsoft, Inc. as an Infrastructure Lead Engineer. In this pivotal role, you will be responsible for leading the design, implementation, and maintenance of our infrastructure systems. You will work closely with cross-functional teams to ensure our infrastructure supports our business objectives efficiently and effectively.As an Infrastructure Lead Engineer, you will leverage your expertise to create robust infrastructure solutions, troubleshoot complex issues, and drive continuous improvement initiatives. Your leadership will guide a team of engineers, fostering a culture of innovation and excellence.

For over three decades, Angi has been at the forefront of the home services sector, creating a dynamic platform where homeowners and professionals thrive through quality work.Homeowners can reliably connect with skilled professionals, while our partners benefit from a robust platform that helps them secure meaningful work opportunities. We are committed to fostering a workplace that our employees are proud to call home.Angi Overview:Originally founded in 1995 as Angie’s List, rebranded in 2021A global enterprise with 9 brands across 8 countries and a diverse workforceTrusted by homeowners for over 300 million home projects and countingTeam Overview:We are looking for a Senior Data Engineer to join our Data Infrastructure team. This role is pivotal in building and maintaining the core platforms that facilitate data processing, storage, and analytics across the organization. You will focus on enhancing our lakehouse architecture, data replication systems, and orchestration frameworks to ensure scalable, reliable, and efficient data workflows.This position is remote, but we prefer candidates located in the Eastern Time Zone to align with our team's working hours.

Vitol is seeking a skilled and driven Python Data Engineer to enhance our data assets and support our analytical initiatives within a full-time capacity. In this role, you will collaborate closely with traders, analysts, researchers, and data scientists to define requirements and fulfill diverse data-related needs.Key Responsibilities:Develop modular, reusable components to facilitate communication between external data sources, internal tools, and databases.Engage with business stakeholders to clarify requirements for data ingestion and accessibility.Convert business requirements into actionable technical solutions.Ensure the integrity and organization of the Vitol Python codebase, adhering to established designs and coding standards.Enhance our developer tools and Python ETL toolkit through standardization and consolidation of core functionalities.Effectively coordinate efforts with our global team.Participate in Vitol’s Python development community and act as a liaison to support our expanding business development initiatives.

At Peregrine Technologies, a company backed by top-tier Silicon Valley investors, we empower public safety organizations, state and local governments, federal agencies, and private-sector entities to tackle societal challenges with unparalleled speed and precision. Our cutting-edge AI-enabled platform transforms fragmented and isolated data into actionable operational intelligence, delivering crucial insights that enhance decision-making processes and improve outcomes across various scenarios. Currently, we proudly serve hundreds of clients across over 30 states and two countries, impacting more than 125 million lives, and we are poised for further growth as we expand into the enterprise sector and international markets.TeamWe believe that empathy is key to enhancing our solutions. Our engineering team prioritizes understanding how users interact with our products, which guides us in finding the best solutions. You'll have the chance to collaborate closely with our onsite team to explore the diverse use cases that Peregrine addresses.We value both ownership and teamwork. In this role, you will be responsible for significant features while working alongside fellow engineers to bring them to fruition. We hold humility and empathy in high regard as essential traits for crafting effective solutions, and you will engage directly with our deployment team and users to iterate on problem-solving. Creativity and resilience are vital as we pursue our vision.RoleWe are in search of a Staff Data Infrastructure Engineer to join our dynamic team. In this role, you will take full ownership of the data layer that is foundational to all of Peregrine's operations. You will design and construct the systems responsible for ingesting, storing, and serving vast amounts of real-time operational data, empowering our clients to make critical decisions quickly and confidently.This senior individual contributor position is ideal for someone who excels at tackling complex technical challenges and possesses the experience and judgement necessary to influence key infrastructure decisions. You will engage with a variety of intricate challenges, including:Designing and managing a high-throughput, real-time data integration platform across diverse customer environments.Architecting a scalable open table format layer to ensure reliable data storage at petabyte scale.Building and optimizing distributed data processing pipelines using Apache Spark and related streaming technologies.Enhancing performance, reliability, and cost efficiency across the entire data infrastructure stack.Collaborating with platform and product engineering teams to define data contracts, schemas, and integration pathways.

Position OverviewAs the Lead Data Engineer, you will spearhead the design and development of advanced, scalable data architectures to facilitate the transition of outdated, file-based analytical systems to modern AWS Cloud Native environments. This pivotal role emphasizes transforming legacy SAS-based data storage models—including flat files, batch outputs, and subsystem-specific data artifacts—into structured, governed, and scalable data frameworks optimized for cloud-native processing.You will ensure data integrity, performance, and visibility across a comprehensive modernization initiative while providing technical leadership in data modeling, ingestion patterns, validation frameworks, and transparency reporting.Expert-level proficiency in Python and substantial experience in architecting AWS-based data solutions are essential for this role.

Join Our TeamWe invite you to become a vital part of Evolv as a Senior Data Infrastructure Engineer within our Machine Learning & Sensors organization. This pivotal role entails the design, construction, and maintenance of robust, secure, and scalable data pipelines that drive our AI/ML research and production systems. You will take charge of the complete data lifecycle—from ingestion across thousands to millions of edge devices, through cloud processing, to a centralized data factory that supports model training, evaluation, and ongoing enhancement.Data is at the core of our mission to revolutionize AI-based weapon detection systems. Your expertise will ensure seamless data flow across various geographies, devices, and cloud systems, while adhering to stringent standards for quality, privacy, security, and scalability. This position is perfect for someone who is passionate about the intersection of distributed systems, cloud pipelines, and ML-driven data requirements.Success in the Role: Your First YearIn the first 30 days:Gain an in-depth understanding of our existing edge-to-cloud data pipelines and deployment environments.Evaluate current data ingestion processes, governance frameworks, and cloud infrastructure.Identify challenges related to data reliability, quality, and operational scalability.Establish rapport with AI/ML, data science, field operations, and cloud engineering teams.Design and prototype both cloud and edge data processing pipelines.Within the first three months:Implement enhancements to critical ingestion, validation, and processing pipelines.Deploy scalable data pipelines using AWS components such as S3, EC2, Lambda, Glue, Step Functions, and SageMaker integrations.Develop automated validation workflows to identify data corruption, missing metadata, or malformed data.Create automated model evaluation, training, and improvement pipelines to accelerate experimentation.Collaborate with field operations to enhance data reliability, observability, and coverage.By the end of the first year:Oversee the entire lifecycle of mission-critical data pipelines that support AI/ML research and production.Architect advanced edge-to-cloud data systems capable of scaling across millions of devices.Establish and enforce data governance frameworks, including retention, access control, privacy, and lineage.Enable ML teams to quickly conduct experiments with high-quality, discoverable, versioned datasets.

At Angi®, our mission for the past 30 years has been simple: to ensure that jobs are done right. We connect homeowners with trustworthy professionals who possess the necessary skills, while simultaneously linking these pros with homeowners seeking the jobs they desire.Angi at a glance:Homeowners have relied on Angi for over 300 million projects.We cover more than 1,000 home service tasks.Our team consists of 2,800 dedicated employees worldwide.Why join Angi:Angi® is on a mission to redefine the home services industry, fostering an environment where homeowners, professionals, and employees all benefit from a greater number of jobs completed successfully.For homeowners, our platform offers a dependable way to locate skilled professionals. For professionals, we act as a trustworthy business partner, helping them discover the work they want when they want it. For our employees, we provide an exceptional workplace that they can proudly call home. We look forward to welcoming you!About the team:We are currently searching for a Senior Data Engineer to join our Data Infrastructure team. This individual will play a pivotal role in constructing and managing the foundational platforms that facilitate data processing, storage, and analytics throughout our organization. The focus of this role will be on advancing our lakehouse architecture, data replication systems, and orchestration frameworks, all while ensuring scalable, reliable, and efficient data workflows.Please note, although this role is remote, we are a global company seeking candidates located in the Eastern Time Zone to align with our team's working hours.

We are seeking a top-tier Data Engineer to join our team at wizdaa. If you are a developer who excels in:Leading your team with technical expertiseResolving complex challenges that others find difficultDelivering intricate features at an accelerated paceCreating exceptionally clean and maintainable codeEnhancing our codebase with pride and diligenceYour skills and experience will help us drive efficiency and innovation in data processing.Key Responsibilities:Develop, enhance, and scale data pipelines and infrastructure utilizing Python, TypeScript, Apache Airflow, PySpark, AWS Glue, and Snowflake.Design, implement, and monitor data ingestion and transformation workflows, ensuring optimal performance and reliability.Work collaboratively with platform and AI/ML teams to automate data workflows and develop a comprehensive feature store.Integrate health metrics into engineering dashboards for enhanced visibility and operational insight.Model data and execute scalable transformations in Snowflake and PostgreSQL.Create reusable frameworks and connectors to streamline internal data processes.

Peregrine Technologies, backed by top-tier Silicon Valley investors, empowers public safety organizations, government entities at all levels, and private institutions to effectively tackle societal challenges with unmatched speed and precision. Our innovative AI-driven platform transforms fragmented data into actionable insights, enabling rapid and informed decision-making that enhances outcomes across the board. Currently, we serve hundreds of clients in over 30 states and two countries, impacting more than 125 million lives as we broaden our reach into enterprise solutions and international markets.Our Engineering TeamWe are a team that prioritizes empathy in our engineering solutions. Understanding how our users interact with our products is essential to our process. Engineers will have the chance to collaborate closely onsite, gaining insights into the diverse use cases our platform addresses.We cherish both ownership and teamwork—you will take full accountability for significant features while working alongside fellow engineers to see them through to completion. We believe humility and empathy are crucial for crafting effective solutions, and you will engage directly with our deployment team and users to iterate and resolve their challenges. Creativity and determination are vital as we pursue our ambitious goals.Your RoleWe are seeking a Staff Data Infrastructure Engineer to join our dynamic team, where you will have substantial ownership over the data ecosystem that supports all of Peregrine's operations. You will design and develop systems that manage, store, and deliver vast amounts of real-time operational data, enabling our customers to make crucial decisions quickly and confidently.This position is ideal for a seasoned individual contributor who excels at solving complex technical challenges and possesses the expertise to influence foundational infrastructure strategies. You will engage with a variety of intricate issues, including:Creating and managing a high-throughput, real-time data integration platform across varied customer environments.Developing a scalable open table format layer for dependable data storage at a petabyte scale.Building and fine-tuning distributed data processing pipelines utilizing Apache Spark and related streaming technologies.Enhancing performance, reliability, and cost-effectiveness throughout the entire data infrastructure stack.Collaborating with platform and product engineering teams to establish data contracts, schemas, and integration pathways.

Seer Interactive is a pioneering digital marketing consultancy dedicated to leveraging data, technology, compassion, and AI to positively impact our communities, including coworkers, clients, and neighbors. We are committed to uncovering and sharing the truth through the integration of millions of data points from both quantitative and qualitative sources, supported by responsible and human-centered AI. Our mission is to unlock potential for our communities, envisioning a world where informed decision-making is second nature.At Seer, we view AI as a transformative force for innovation, collaboration, and creativity. Regardless of your department, utilizing AI is integral to how we achieve smarter, faster, and more strategic outcomes for our clients. Our AI Council facilitates company-wide integration through education, purposeful implementation, and ongoing learning — ensuring AI becomes an embedded aspect of our operations and growth.If you are prepared to elevate others, challenge your own biases, commit to truth, and strive for continuous improvement—read on.Seer is a remote-first agency and a Certified B-Corp with team members nationwide. While this position is primarily remote, those who prefer an office environment can work from our headquarters in Philadelphia. We are only able to consider candidates located in the US who can operate during continental US hours.The Role: As the Data Engineering Lead at Seer, you will be the pivotal force behind our data platform. You will establish architectural directions, mentor fellow engineers, and maintain a hands-on approach by actively contributing code, not just reviewing it. Your leadership will be characterized by influence, expertise, and trust rather than direct reporting lines.At Seer, we integrate AI into our processes to provide more intelligent, rapid, and strategic solutions, as the modern marketing data landscape continues to evolve beyond traditional dashboards. You will play a crucial role in this evolution, designing platforms that enable strategists, analysts, and LLMs to query a unified source of truth and derive superior solutions more quickly. Additionally, you will elevate the engineering standards for your peers through mentorship, code reviews, and collaborative efforts.

Join Mindrift as a Freelance Python Data Scraping Engineer, where you'll be pivotal in the Tendem project, orchestrating sophisticated data scraping workflows within our innovative hybrid AI and human system.In this role, referred to as an AI Pilot at Mindrift, you will work in tandem with Tendem Agents who manage repetitive tasks. Your role will focus on applying your critical thinking, domain expertise, and quality control skills to ensure the delivery of accurate and actionable results.This part-time remote opportunity is perfect for technical professionals with a solid background in web scraping, data extraction, and processing.About MindriftMindrift connects specialists with groundbreaking AI projects from leading tech innovators. Our mission is to unleash the potential of Generative AI by leveraging real-world expertise from around the globe.Role OverviewThis freelance position involves executing data scraping tasks that demand technical precision for web extraction and processing. You will utilize various tools, including Apify and OpenRouter, alongside your creative problem-solving methods.Key ResponsibilitiesManage end-to-end data extraction workflows across complex websites, ensuring comprehensive coverage, accuracy, and reliable delivery of structured datasets.Utilize internal tools (Apify, OpenRouter) along with custom workflows to expedite data collection, validation, and task execution while adhering to defined requirements.Ensure dependable extraction from dynamic and interactive web sources, adjusting strategies as necessary to accommodate JavaScript-rendered content and evolving site behaviors.Implement data quality standards through validation checks, cross-source consistency controls, adherence to formatting specifications, and systematic verification prior to delivery.Scale scraping operations for large datasets using efficient batching or parallelization, monitor failures, and maintain stability amidst minor site structure changes.CompensationContributors to this project can earn up to $32 per hour, depending on their contribution level and pace. Compensation may vary across different projects based on scope, complexity, and expertise required.How to ApplyApply to this posting, qualify, and seize the opportunity to engage in projects that align with your technical skills on your own schedule. From coding and automation to fine-tuning AI outputs, your role will be instrumental in advancing AI capabilities and real-world applications.RequirementsA minimum of 3 years of relevant experience in data engineering, web scraping, automation, or a related field.

The Bot CompanyAt The Bot Company, we are on a mission to create an innovative robot that enhances everyday life in homes everywhere.Located in the heart of San Francisco, our compact team comprises talented engineers, designers, and operators hailing from esteemed organizations such as Tesla, Cruise, OpenAI, Google, and Pixar. With a track record of delivering exceptional products to hundreds of millions of users, we understand the intricacies involved in crafting remarkable experiences.Our intentionally lean structure fosters swift decision-making while eliminating unnecessary bureaucracy. Each team member operates as an individual contributor, endowed with substantial autonomy, ownership, and accountability. We thrive on a culture of rapid iteration and efficient execution, working collaboratively across the technology stack.

Position OverviewJoin OpenEvidence as a Data Infrastructure Software Engineer, where you will engineer comprehensive systems that drive essential product and research operations. Your focus will be on optimizing performance, ensuring scalability, and enhancing accuracy, while enjoying the autonomy to manage the infrastructure that assists healthcare professionals in navigating complex clinical decisions in real-time.We value exceptional creators who thrive in versatile roles. Our engineers engage across various products and projects, taking ownership wherever they can make the most significant impact.About OpenEvidenceOpenEvidence is the leading medical AI platform globally, utilized by over 40% of clinicians in the U.S. in just over a year through organic product-led growth. As a $12 billion company, our engineering team comprises 30 talented individuals from MIT, Harvard, and Stanford. We believe that groundbreaking products are born from a small group of exceptional builders, driven by focused goals and empowered to take ownership and act swiftly. We are expanding our team to capitalize on an unparalleled opportunity to set the standard for medical AI platforms.If you are a top-tier engineer or scientist eager to push the boundaries and achieve tangible outcomes that affect millions of lives, we want to connect with you.Our CultureWe expect our work to be performed at an elite level. The journey from concept to execution and scaling is akin to a professional sport, where excellence is non-negotiable. We believe that the creation of innovative technologies is only achievable through complete ownership. Significant achievements happen when individuals take the initiative to see them through.Your ProfileThis role is not for those seeking a 9-to-5 job or merely looking to write papers. If you are ready to dive into the trenches, tackle challenges head-on, and create something from scratch that could impact millions and drive substantial revenue, you might be the perfect fit.We seek brilliant builders who are intelligent, ambitious, resourceful, self-reliant, detail-oriented, driven, hardworking, and humble. Does this sound rare? It is, as we have only found 30 of them so far, and we are eager to discover more.