Data Engineer

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Data Engineer based in South Africa.

This role offers the opportunity to help build, maintain, and modernize a growing data platform in a cloud-first environment. You will design reliable data pipelines and scalable data solutions that enable teams to access and use high-quality information effectively. The role combines modern AWS infrastructure with opportunities to improve and transition legacy data processes. You will work across batch and real-time data workflows, APIs, integrations, and data warehouse solutions. Collaboration with analytics teams and other stakeholders will be central to understanding requirements and delivering practical solutions. This is a hands-on opportunity to improve data reliability, performance, security, and quality while contributing to the evolution of the broader data architecture.

Accountabilities

• Design, develop, and maintain scalable and reliable data pipelines and Extract, Transform, Load (ETL) and Extract, Load, Transform (ELT) processes.
• Develop, maintain, and optimize data warehouse solutions and data models.
• Build and maintain data infrastructure primarily on Amazon Web Services (AWS), while supporting relevant legacy systems.
• Develop APIs and data integrations to enable effective data exchange across systems.
• Contribute to real-time data pipelines and processing using Apache Kafka.
• Automate and optimize data processes, workflows, and operational tasks.
• Monitor, troubleshoot, and continuously improve the reliability, performance, security, and data quality of pipelines and infrastructure.
• Work closely with analytics teams and other stakeholders to understand data requirements and deliver effective technical solutions.
• Maintain accurate and up-to-date documentation covering pipelines, infrastructure, workflows, and operational processes.
• Contribute to the modernization of legacy data processes, infrastructure, and overall data architecture.

Requirements

• 2+ years of professional experience in data engineering.
• Strong SQL and Python programming skills.
• Hands-on experience with AWS services, particularly Amazon S3, Amazon Redshift, AWS Glue, and AWS Lambda.
• Practical experience with Apache Airflow for data workflow orchestration.
• Strong understanding of data warehousing principles and data modeling.
• Experience designing, developing, and maintaining production-grade data pipelines, with a strong focus on reliability, monitoring, and data quality.
• Solid understanding of both relational and non-relational databases.
• Familiarity with Git and Continuous Integration/Continuous Deployment (CI/CD) practices, preferably using GitLab.
• Experience developing APIs and building data integrations.
• Strong written and verbal communication skills.
• Upper-intermediate English proficiency or higher.
• Experience with Kafka and real-time data processing is a plus.
• Experience with Microsoft SQL Server and SQL Server Integration Services (SSIS) is a plus.

Benefits

• Full-time position.
• Opportunity to work within an Analytics-focused environment.
• Remote working arrangement.
• Opportunity to contribute to modern cloud-based data infrastructure.
• Hands-on exposure to AWS, Airflow, Kafka, data warehousing, and data modernization initiatives.
• Opportunity to contribute to the improvement of data reliability, quality, performance, and architecture.
• Collaborative work with analytics teams and cross-functional stakeholders.