Gapstars logoGapstars logo

Lead Data Engineer / Associate Architect at Gapstars | Colombo

GapstarsColombo, Western Province, Sri Lanka
On-site Full-time

Clicking Apply Now takes you to AutoApply where you can tailor your resume and apply.


Experience Level

Manager

Qualifications

The ideal candidate will have:Proven experience in data engineering, particularly in cloud environments, with a focus on Azure Databricks. Strong understanding of data architectures, data modeling, and ETL processes. Experience with data warehousing solutions and data pipeline development. Proficiency in programming languages such as Python, SQL, and experience with tools like Azure Data Factory. Excellent problem-solving skills and the ability to communicate complex technical concepts to non-technical stakeholders.

About the job

About Gapstars

Gapstars partners with some of Europe’s most ambitious tech companies, from disruptive startups to rapidly growing scaleups. Our mission is to assist these companies in building high-performing remote engineering teams. Headquartered in the Netherlands, with talent hubs in Sri Lanka and Portugal, we proudly host over 275 engineers who excel at solving real-world challenges using cutting-edge technologies. Our teams operate across various domains, including networking, marketplaces, SaaS, and AI, delivering scalable solutions that produce meaningful outcomes. If you desire a workplace that champions technical excellence, fosters a strong culture, and offers opportunities for growth, look no further than Gapstars.

The Role

As we transition our Data and Analytics operations entirely to the cloud (specifically Azure Databricks), we are on the lookout for a Lead Data Engineer/Architect to lead our data engineering and architecture initiatives. This role will involve collaborating closely with the Head of Data and Analytics and guiding a team of master data specialists, analytics/BI engineers, reporting developers, and data scientists in converting business requirements into efficient end-to-end solutions. You will also determine where we can add the most value within our highly engaged organization.

Your responsibilities will include working with diverse data sets from multiple systems, ensuring their seamless integration and optimization to extract valuable insights for our business. This will cover everything from the initial development of data pipelines to their ongoing management and optimization using tools available in Azure Cloud.

Your contributions will directly influence the creation and implementation of data-driven strategies, providing pivotal insights that enhance decision-making and strategic planning. Join us in our mission to revolutionize urban living by ensuring our data management and analysis processes are efficient, reliable, and insightful.

The platform you'll inherit:

  • Azure Databricks on Private PaaS - VNet injection, no public IPs, 57 private endpoints, NAT gateways with stable egress

  • 5-subscription CAF Landing Zone architecture with full network isolation

  • Metadata-driven ingestion via ADF orchestrating Databricks notebooks, with Autoloader and DLT pipelines downstream

  • 40+ data sources including Infor (global ERP), Xero (7 regions), Cin7 (4 regions), NetSuite, Salesforce, BigCommerce, Lightspeed, and others

  • Unity Catalog, Key Vault integration, CI/CD from Azure DevOps

  • Annual Databricks commitment on a 3-year contract

The reality you'll walk into:

  • Bronze and silver layers exist for most sources, but the gold layer is under development and presents an exciting opportunity for growth.

About Gapstars

Gapstars is committed to fostering innovation and collaboration, making it a vibrant environment for engineers to thrive. The company embraces cutting-edge technologies and is dedicated to building a diverse and inclusive workforce, making it a perfect fit for professionals who value growth and teamwork.

Similar jobs

Browse all companies, explore by city & role, or SEO search pages. View directory listings: all jobs, search results, location & role pages.

Tailoring 0 resumes

We'll move completed jobs to Ready to Apply automatically.