Clear Tech logoClear Tech logo

Semi Senior Data Engineer - Azure Databricks

Clear TechRemote Latam
Remote Full-time

Clicking Apply Now takes you to AutoApply where you can tailor your resume and apply.


Experience Level

Mid to Senior

Qualifications

3+ years of experience in Data Engineering. Proven experience in Azure Databricks. Strong proficiency in Python and SQL. Experience in building ETL pipelines with PySpark and Delta Lake. Hands-on experience with Azure Data Factory.

About the job

At Clear Tech, we are pioneers in harnessing the power of Data, Analytics, and Artificial Intelligence. By empowering companies globally, we transform raw data into meaningful business insights. Our team, comprised of highly skilled professionals from Latin America, follows global best practices in cloud technologies, data engineering, data science, and business intelligence. We adopt agile methodologies to execute comprehensive projects, staff augmentation models, and tailored training programs, equipping professionals to meet modern market challenges.

 

Semi Senior Data Engineer - Azure Databricks

We are on the lookout for a Semi Senior Data Engineer with robust expertise in Azure Databricks. This role involves designing, developing, and maintaining scalable data pipelines within a state-of-the-art Lakehouse architecture.

Your responsibilities will heavily focus on data engineering within Azure, including orchestration with Azure Data Factory, storage management, and ETL development using Python and Databricks, alongside governance through Unity Catalog.

We seek a hands-on individual who possesses excellent skills in Python and SQL and is comfortable working with large data volumes in distributed environments.

Responsibilities

  • Design and implement scalable ETL/ELT pipelines utilizing Azure Databricks and PySpark.
  • Develop robust data transformation solutions using Python and SQL.
  • Create and maintain Databricks notebooks for ingestion and transformation processes.
  • Orchestrate data flows using Azure Data Factory.
  • Manage governance and access control through Databricks Unity Catalog.
  • Establish and oversee Databricks Jobs for production loads.
  • Work with Azure storage solutions (Data Lake, Blob Storage, etc.).
  • Optimize queries and pipelines to enhance performance and scalability.
  • Ensure data quality, consistency, and reliability.
  • Collaborate with analytics and business teams to enable data-driven solutions.
  • Document technical processes, architecture, and deployments.
  • Leverage AI tools to enhance development, debugging, and technical productivity.

Main Requirements

  • 3+ years of experience in Data Engineering.
  • Proven experience in Azure Databricks.
  • Strong proficiency in Python and SQL.
  • Experience in building ETL pipelines with PySpark and Delta Lake.
  • Hands-on experience with Azure Data Factory.

About Clear Tech

At Clear Tech, we excel in transforming data into actionable business insights through our expertise in Data, Analytics, and Artificial Intelligence. Leveraging the talent of our team in Latin America and adhering to global best practices, we offer comprehensive solutions in cloud technologies, data engineering, and business intelligence, focusing on agility and excellence in every project.

Similar jobs

Browse all companies, explore by city & role, or SEO search pages.

Tailoring 0 resumes

We'll move completed jobs to Ready to Apply automatically.