Senior Data Engineer - ETL & Pipelines

About the Client

Our client operates a high-volume B2C creator platform in the entertainment industry, serving more than 200,000 active users and 30,000 creators, with payment events flowing through the platform every second.

Data is one of the company’s most valuable competitive advantages. In this role, you will work closely with an experienced data analyst to transform a continuous flow of events into clean, reliable, and query-ready data that supports product, payments, and growth decisions.

The Role

We are looking for a Data Engineer to take end-to-end ownership of the company’s ETL pipelines—the systems responsible for moving data from the transactional platform, including MongoDB, payment streams, and behavioral events, into the analytics environment where it can be turned into actionable insights.

This is not simply a dashboard integration role.

You will design and maintain pipelines that remain accurate, reliable, and timely while millions of events are processed and financial transactions occur every second.

Data integrity is critical. A missing payment event or duplicated transaction is not merely a technical issue—it can have a direct financial impact.

What You’ll Do

• Own the pipelines: Design, build, and maintain ETL and streaming pipelines that ingest high-volume event and payment data and deliver it to the analytics warehouse reliably and accurately.
• Model the data: Partner closely with the Lead Data Analyst to create warehouse schemas that answer essential business questions related to creator earnings, payment throughput, retention, lifetime value, and cohort behavior.
• Engineer for failure: Build idempotent, replayable, and observable jobs that can withstand traffic spikes, retries, and partial failures without compromising downstream data.
• Process high-volume events: Move data through queue infrastructure including BullMQ and Redis, AWS SQS, and RabbitMQ, while instrumenting pipelines through OpenTelemetry.
• Protect data integrity: Implement reconciliation checks, deduplication, backfills, and reliable handling of late or out-of-order events.
• Protect financial transactions: Collaborate with payments and platform teams to ensure that every transaction is captured once, accurately, and on time.

The Technology Stack

• Languages: Node.js and TypeScript as the primary technologies, with some .NET and C#
• Data stores: MongoDB as the primary transactional database and Redis
• Queues and streaming: BullMQ, AWS SQS, and RabbitMQ
• Analytics and warehouse: PostHog, its data warehouse layer, and SQL data modeling
• Platform: AWS, including S3 and SNS, Docker, Kubernetes, and GitHub Actions
• Observability: OpenTelemetry tracing across every pipeline