DE&A - Core - Advanced Visualization & Cloud

About The Role

Role Overview

We are seeking a skilled Data Integration & Middleware Engineer Lead to implement and manage scalable data pipelines and middleware solutions. The role will focus on orchestrating data workflows using Azure technologies such as Azure Data Factory (ADF), Azure Blob Storage, and Azure SQL Database, ensuring efficient ETL/ELT processing and high-quality data delivery.

Key Responsibilities

Manage end-to-end data integration pipelines using Azure Data Factory (ADF)
Orchestrate and monitor data workflows , ensuring reliability, performance, and error handling
Implement\ Manage ETL/ELT processes for data ingestion, transformation, and loading
Manage and optimize data storage solutions using Azure Blob Storage
Maintain structured data models in Azure SQL Database / Azure DB
Ensure data quality, validation, and governance across pipelines
Support integration with on-premise and cloud-based systems
Perform performance tuning and cost optimization across the data platform

Technical Skills Required

Core Skills – Must Have

Strong experience with Azure Data Factory (ADF) for orchestration
Hands-on experience with ETL/ELT development
Experience with Azure Blob Storage / Data Lake
Proficiency in Azure SQL Database / SQL Server
Understanding of data warehousing concepts and data modeling
Knowledge of DevOps, CI/CD pipelines for data workflows

Nice to Have

Experience with Azure Databricks or Synapse Analytics
Knowledge of data pipeline optimization and monitoring
Familiarity with API integrations and data connectors
Exposure to data governance and security practices
Azure certifications (e.g., AZ-900, DP-203 )
Experience with real-time data processing frameworks (Event Hub, Service Bus)

Soft Skills

Strong analytical and problem-solving abilities
Clear communication and stakeholder management skills
Ability to work in a collaborative, cross-functional environment
Attention to detail and commitment to data quality

Key Responsibilities

Manage end-to-end data integration pipelines using Azure Data Factory (ADF)
Orchestrate and monitor data workflows , ensuring reliability, performance, and error handling
Implement\ Manage ETL/ELT processes for data ingestion, transformation, and loading
Manage and optimize data storage solutions using Azure Blob Storage
Maintain structured data models in Azure SQL Database / Azure DB
Ensure data quality, validation, and governance across pipelines
Support integration with on-premise and cloud-based systems
Perform performance tuning and cost optimization across the data platform

Lead and Support Engineer