Sr Azure Data Engineer/Architect

Responsibilities

• Assure that data is cleansed, mapped, transformed, and otherwise optimised for storage and use according to business and technical requirements
• Develop and maintain innovative Azure solutions
• Solution design using Microsoft Azure services and other tools
• The ability to automate tasks and deploy production standard code (with unit testing, continuous integration, versioning etc.)
• Load transformed data into storage and reporting structures in destinations including data warehouse, high speed indexes, real-time reporting systems and analytics applications
• Build data pipelines to collectively bring together data
• Other responsibilities include extracting data, troubleshooting and maintaining the data warehouse

Tech Stack

Python

SQL and NoSQL databases

Scala

Spark-SQL

Experience with Azure: ADLS, Databricks, Stream Analytics, SQL DW, COSMOS DB, Analysis Services, Azure Functions, Serverless Architecture, ARM Templates

Qualifications

• Previous experience as an Azure Data Engineer or similar role
• Experience building and optimising ‘big data’ data pipelines, architectures, and data sets
• Strong analytic skills related to working with unstructured datasets
• The ability to design and implement well written code
• Ability to work to tight deadlines
• Ability to test the data from source to the presentation layer
• Ability to support/troubleshoot data pipelines
• Confident and concise communication skills, with the ability to drive alignment, collaboration, and efficiency within teams