About the job
We invite you to submit your CV in English and specify your English proficiency level.
Mindrift is dedicated to bridging specialists with project-oriented AI roles within top tech firms, emphasizing the testing, evaluation, and enhancement of AI systems. Note: Participation is project-based, not permanent employment.
Opportunity Overview
In this role, you will design rigorous coding test cases that challenge AI coding systems to their utmost limits:
- Critically assess and enhance realistic coding tasks derived from existing production codebases, ensuring they are aligned with genuine scope, requirements, and information sources.
- Develop thorough functional tests that confirm actual end-to-end behavior and edge cases, surpassing mere superficial checks.
- Create “fair but tough” challenges where the AI possesses all necessary context but must exert effort to uncover it (information may be dispersed across files and external sources, requiring complex reasoning).
- Examine AI failures to discern the model's weaknesses versus its strengths.
- Refine your approach based on evaluations from expert QA reviewers who assess your work against seven quality criteria.
Ideal Candidate Profile
This position is ideal for seasoned developers, software engineers, and/or test automation specialists looking for part-time, non-permanent projects. Preferred qualifications include:
- A degree in Computer Science, Software Engineering, or related discipline.
- 5+ years of experience in software development, predominantly using Python (pytest, async/await, subprocess, file operations).
- A strong foundation in Full-Stack development, with balanced expertise in creating React-based interfaces and robust back-end systems.
- Proficiency in writing tests (functional, integration) rather than just executing them.
- Experience with Docker containers (facilitating local evaluations within containers).
- Understanding of CI/CD processes (familiarity with GitHub Actions: triggers, labels, interpreting results).
- English proficiency at B2 level or higher.
Working Process
Application → Qualification → Project Involvement → Task Completion → Compensation
Estimated Workload
Tasks are projected to require around 20 hours for completion, contingent on complexity. This is an estimation rather than a strict schedule; you have the flexibility to choose when and how to work. Submissions must meet deadlines and acceptance criteria to be considered complete.
Compensation
Contributors can earn up to $24 per hour, depending on their expertise and contribution pace. Compensation varies by project based on its scope, complexity, and required skill set. Note that other projects may offer different compensation levels based on specific requirements.

