About the job
Join Mindrift as a Freelance Python Data Scraping Engineer!
We are seeking highly skilled Data Scraping Engineers to contribute to our Tendem project, which integrates advanced AI and human workflows for specialized data scraping tasks.
In this unique role, referred to as an AI Pilot at Mindrift, you will work alongside Tendem Agents who manage repetitive tasks, while you apply your critical thinking, domain expertise, and quality control skills to ensure the delivery of precise and actionable data.
This part-time, remote position is perfect for technical experts experienced in web scraping, data extraction, and processing.
About Mindrift
The Mindrift platform connects skilled professionals with cutting-edge AI projects from leading tech innovators. Our goal is to harness the power of Generative AI by leveraging real-world expertise from around the world.
Your Role
As a Python Data Scraping Engineer, you will manage the data scraping process, ensuring technical accuracy in web extraction and processing. You will utilize tools such as Apify and OpenRouter, as well as your own innovative methods to achieve optimal results.
Core Responsibilities
- Oversee end-to-end data extraction workflows for complex websites, guaranteeing complete coverage, precision, and reliable delivery of structured datasets.
- Utilize internal tools (Apify, OpenRouter) in conjunction with custom solutions to enhance data collection, validation, and task execution while adhering to specified requirements.
- Adapt scraping techniques to reliably extract data from dynamic and interactive web sources, accommodating JavaScript-rendered content and evolving site behaviors.
- Maintain high data quality standards through validation checks, consistency controls from multiple sources, formatting adherence, and systematic verification prior to data delivery.
- Scale scraping capabilities for extensive datasets using efficient batching or parallelization, monitor failures, and ensure stability despite minor changes in site structure.
Compensation
Contributors on this project can earn up to $32 per hour, depending on their experience and contribution pace. Compensation may vary across projects based on their scope and complexity.
How to Apply
To get started, simply apply to this posting, qualify, and have the opportunity to work on projects that align with your technical skills, all on your own schedule. Your contributions will be vital in advancing AI capabilities and real-world applications.
Qualifications
- At least 3 years of relevant experience in data engineering, web scraping, automation, or related fields.
- Proficiency in Python and web scraping frameworks.
- Strong analytical and problem-solving skills.
- Ability to work independently and manage multiple projects concurrently.
