[Wattpad] Data Engineer - Maternity Cover

What You'll Do

-

Coordinate with data scientists, data analysts, software engineers and product managers to design and evolve our existing architecture, improving the reliability and performance of our data infrastructure

-

Build, maintain and scale our ETL pipelines to help power data science, data products and analytical needs

-

Design, build, and maintain data infrastructure systems such as but not limited to AWS DMS, HBase, EMR, Redshift, S3, Kafka, Airflow while ensuring scalability, reliability, and security

-

Write new tools, services, and platforms in Python

-

Research new technologies to solve tomorrow’s data infrastructure needs

-

Advocate for the adoption of tools in the data engineering team

Required Qualifications

• -
• 4+ years of data engineering experience
• -
• Ability to write high-quality code in Python and PySpark
• -
• Strong understanding of database fundamentals and software engineering principles
• -
• Strong understanding of Data Contracts, microservices principles, and treating datasets/platforms as API-driven products
• -
• Deep understanding of two or more of these technologies: Spark, Airflow, AWS EMR, AWS Glue, Sagemaker and Redshift
• -
• Experience with batch processing and streaming systems
• -
• Exposure and/or strong interest in Kafka or other queuing technologies
• -
• A critical thinker and possess creative problem solving skills
• -
• Quantitative skills, ability to think analytically, and strong attention to details
• -
• A desire to contribute to the broader development community