Lead Data Engineer - Microsoft Fabric
BrillioChennai, Tamil Nadu, India
On-site Full-time
Clicking Apply Now takes you to AutoApply where you can tailor your resume and apply.
Experience Level
Manager
Qualifications
Qualifications7+ years of extensive experience in data engineering, particularly with Microsoft Fabric and associated technologies.
About the Role
Join our innovative team at Brillio as a Lead Data Engineer specializing in Microsoft Fabric. We are looking for a seasoned professional with over 7 years of experience in data engineering. In this pivotal role, you will spearhead the design, development, and deployment of data pipelines, optimizing data movement and integration within the Microsoft Fabric ecosystem.
Key Responsibilities
- Data Pipeline Development: Create, develop, and implement data pipelines using Microsoft Fabric, incorporating OneLake, Data Factory, and Apache Spark for efficient, scalable, and secure data operations.
- ETL Architecture: Design and execute ETL workflows tailored for Fabric’s integrated data platform to enhance data ingestion, transformation, and storage processes.
- Data Integration: Develop and maintain solutions that consolidate both structured and unstructured data sources into Fabric’s OneLake environment, utilizing SQL, Python, Scala, and R for advanced data handling.
- Fabric OneLake & Synapse: Utilize OneLake as the central data repository to facilitate enterprise-level analytics, seamlessly working with Synapse Data Warehousing for comprehensive big data processing and reporting.
- Cross-functional Collaboration: Collaborate with Data Scientists, Analysts, and BI Engineers to ensure that Fabric’s data infrastructure effectively supports Power BI, AI workloads, and advanced analytics.
- Performance Optimization: Oversee, troubleshoot, and enhance Fabric pipelines to ensure high availability, rapid query performance, and minimized downtime.
- Data Governance & Security: Enforce governance and compliance frameworks within Fabric, ensuring data lineage, privacy, and security across the unified platform.
- Leadership & Mentorship: Lead and mentor a talented team of engineers, overseeing Fabric workspace design, code reviews, and the implementation of new Fabric features.
- Automation & Monitoring: Automate workflows using Fabric Data Factory, Azure DevOps, and Airflow to ensure operational efficiency.
- Documentation & Standards: Thoroughly document Fabric pipeline architecture, data models, and ETL processes while contributing to engineering best practices and enterprise guidelines.
- Innovation: Stay abreast of Fabric’s evolving functionalities (such as Real-Time Analytics and AI integration) and foster a culture of innovation within the team.
About Brillio
Brillio is a transformative digital consulting firm that empowers organizations to harness data and technology, delivering innovative solutions that drive business growth and efficiency.
Similar jobs
Browse all companies, explore by city & role, or SEO search pages.
