About the job
Zyphra is a cutting-edge artificial intelligence firm located in the heart of San Francisco, California, dedicated to advancing technology across various modalities.
About the Position:
We are seeking a Data Engineer - Multimodal Systems to play a pivotal role in the enhancement and expansion of Zyphra's datasets and data pipelines. This position offers a unique opportunity to collaborate with diverse teams and contribute to innovative data solutions. You will engage in the collection of extensive datasets and the development and optimization of high-performance parallel data pipelines.
Your Responsibilities Will Include:
Executing large-scale data collection across multiple modalities, including text, audio, and image.
Designing and implementing highly efficient, parallelized data processing pipelines that integrate various modalities.
Conducting rigorous experimental ablations to evaluate the effectiveness of new data enhancements.
Candidate Requirements:
Proven ability in implementation and prototyping.
Capability to transform ideas into experimental frameworks swiftly.
Strong collaborative skills, thriving in a dynamic research environment.
Eagerness to learn and apply new concepts effectively.
Exceptional communication and teamwork skills, capable of contributing to both research and large-scale engineering projects.
Preferred Qualifications:
Experience in the collection, management, and processing of large datasets.
Familiarity with parallel programming frameworks in Python, such as Dask.
In-depth understanding of state-of-the-art dataset curation practices.
A detail-oriented mindset with a passion for data integrity and verification.
Strong foundation in experimental methodologies for conducting thorough ablation studies and hypothesis testing.
Knowledge and interest in large-scale, highly parallel data processing systems.
Proficiency in PyTorch and Python.
Experience with large, complex codebases and the ability to quickly become productive within them.
Published research in respected machine learning venues.
Postgraduate degree in a relevant field is a plus.
