IN_Senior Associate_ Azure Data Engineer _Data and Analytics_Advisory_Kolkata

Job Description & Summary: We are seeking an experienced Azure Data Engineer to design, develop, and optimize data solutions on the Microsoft Azure platform. The role involves building scalable data pipelines, implementing modern data architectures, and ensuring reliable data movement and transformation across Azure's data ecosystem including ADF, Synapse, ADLS Gen2, and Microsoft Fabric. Job Position Title: IN_Senior Associate_ Azure Data Engineer _Data and Analytics_Advisory_Kolkata Responsibilities: Design and develop end-to-end data pipelines using Azure Data Factory (ADF) — linked services, datasets, pipelines, triggers, and parameterization. Implement data lake solutions on ADLS Gen2 — folder structures, partitioning strategies, file formats (Parquet/Delta/CSV), and lifecycle management. Build analytical workloads using Azure Synapse Analytics — dedicated/serverless SQL pools, Spark pools, and Synapse Pipelines. Develop and maintain databases, stored procedures, and optimized queries in Azure SQL. Build and manage data engineering workloads in Microsoft Fabric — Lakehouses, Warehouses, Data Pipelines, Notebooks, and Dataflows Gen2. Write advanced SQL for complex transformations, MERGE operations, window functions, CTEs, and performance tuning. Develop Python scripts for data transformations, API integrations, and automation tasks. Leverage PySpark within Synapse/Fabric Spark pools for large-scale data processing when required. Implement incremental load patterns, CDC, watermarking, and error handling within pipelines. Monitor pipeline performance, troubleshoot failures, and implement alerting mechanisms. Mandatory skill sets: Azure Data Factory (Must-Have): 3+ years — Pipeline design, triggers, parameterization, linked services, IR configuration, CI/CD ADLS Gen2 (Must-Have): Data lake design, folder/partition strategies, file formats, access control (ACLs, RBAC) Azure Synapse (Must-Have): Dedicated & serverless SQL pools, Spark pools, external tables, Synapse Pipelines Azure SQL (Must-Have): Database development, stored procedures, indexing, query optimization Microsoft Fabric (Must-Have): Lakehouses, Warehouses, Pipelines, Notebooks, Dataflows Gen2 SQL (Must-Have): Advanced — MERGE, window functions, CTEs, dynamic SQL, performance tuning Python (Must-Have): Intermediate — scripting, data manipulation (Pandas), API calls, automation PySpark (Must-Have): DataFrame operations, transformations within Synapse/Fabric Spark Preferred skill sets: Experience with CI/CD for ADF/Synapse (Azure DevOps, ARM templates, Git integration). Familiarity with data modeling concepts (Star Schema) Exposure to Power BI for reporting and data visualization. Knowledge of Azure security — Key Vault, Managed Identities, Private Endpoints. Experience with Event Hub / Service Bus for event-driven architectures. Microsoft certifications (DP-203: Azure Data Engineer Associate, DP-600: Fabric Analytics Engineer) are a plus. Exposure to Agile/Scrum delivery methodologies. Years of experience required: · 4- 7 years Education qualification: · BE/ Btech/ Mtech/ MCA/ MBA (fulltime)