About the job
nimblegravity is seeking an experienced Senior Data Engineer to design and elevate robust data solutions that meet critical business objectives. In this pivotal role, you will focus on creating efficient data pipelines, ensuring reliable data transformations, and delivering fully functional systems ready for production use. The successful candidate will possess a strong background in Python, SQL, Databricks/PySpark, and contemporary data engineering methodologies.
Key Responsibilities
- Develop, enhance, and manage scalable data solutions that align with business needs.
- Design and optimize high-performance data pipelines, including extraction, loading, transformation, and orchestration, ensuring they are scalable, reliable, and maintainable.
- Advocate for modern software engineering methodologies including CI/CD, infrastructure as code, containerization, and cloud-native deployments.
- Engage with business stakeholders to convert use cases into production-ready services and solutions, owning the system from inception through to production.
- Establish rigorous testing and monitoring processes to guarantee high data quality and integrity.
Qualifications
Education & Certifications
- A Bachelor’s degree or higher in a STEM field is required.
- A concentration in Computer Science, Mathematics, Physics, or a related engineering discipline is preferred.
Professional Experience
- 5+ years of experience in data engineering or a closely related field, showcasing a proven track record.
Skills & Attributes
- Proficiency in Python and SQL, with a solid understanding of data manipulation and analysis.
- Skilled in Databricks/PySpark and dbt for data warehousing and transformation tasks.
- Familiarity with workflow orchestration tools such as Airflow or Dagster.
- Experience with large language models (LLMs), especially in areas like prompt engineering, retrieval-augmented generation (RAG), and vector databases is advantageous.
- Understanding of machine learning fundamentals, feature engineering, and knowledge graphs is a plus.
- Proven capability in designing and implementing complex data systems from the ground up.
- Expert in managing large-scale data projects, including data cleansing, ETL processes, and information retrieval.
