IN_Senior Associate_Azure Databricks_D&A_Advisory_PAN India

Job Description & Summary: PwC India is looking for a skilled and motivated Senior Associate specializing in Azure Databricks to join our Data & Analytics team. This role involves designing, developing, and optimizing large-scale data processing and analytics solutions leveraging Azure Databricks and related cloud technologies. The ideal candidate will work closely with cross-functional teams to build scalable data pipelines, implement ETL processes, and enable data-driven decision-making across the enterprise. Job Position Title: IN_Senior Associate_Azure Databricks_D&A_Advisory_PAN India Responsibilities: Design, develop, and maintain scalable data pipelines and ETL processes using Azure Databricks and Apache Spark. Collaborate with data engineers, data scientists, and business stakeholders to understand data requirements and translate them into efficient technical solutions. Optimize performance of Databricks clusters and Spark jobs for cost-efficiency and speed. Implement data integration and transformation workflows from diverse data sources in cloud environments (Azure Data Lake Storage, Azure SQL, Cosmos DB, etc.). Support deployment and orchestration of data workflows using Azure Data Factory or similar tools. Ensure data quality, integrity, and security in all data processes. Assist in troubleshooting and resolving issues related to data ingestion, processing, and analytics. Participate in code reviews, documentation, and knowledge sharing within the team. Stay updated with latest Azure cloud and big data technologies and recommend improvements. Mandatory skill sets: Hands-on experience with Azure Databricks, Apache Spark, and PySpark or Scala programming. Strong understanding of data engineering concepts, ETL/ELT processes, and big data architectures. Proficiency in SQL for data querying and manipulation. Experience with Azure services including Azure Data Lake Storage, Azure Data Factory, Azure SQL Database. Knowledge of cloud data security and governance best practices. Familiarity with version control systems such as Git. Ability to write optimized and maintainable code adhering to best practices. Good analytical and problem-solving skills with a collaborative mindset. Preferred skill sets: Experience with streaming data processing (Structured Streaming, Kafka) on Databricks. Knowledge of containerization and orchestration tools like Docker and Kubernetes. Exposure to DevOps practices for data pipelines deployment and monitoring. Familiarity with machine learning workflows using MLflow or similar platforms in Databricks. Understanding of data modeling techniques and dimensional modeling in a cloud environment. Prior experience working with Agile methodologies and tools like Jira. Years of experience required: 3-8 Years Education qualification: BE, <B.Tech>, MCA, <M.Tech>, or equivalent degree with a strong background in computer science or engineering.