Staff Engineer (Distributed Storage, HPC & AI Infrastructure)
Together AI · Amsterdam, Netherlands
About The Role
Join our team as a Staff Engineer specializing in Distributed Storage, HPC, and AI Infrastructure. In this role, you will design and deliver multi-petabyte storage systems for AI training and inference workloads. You will architect high-performance parallel filesystems and object stores, evaluate and integrate cutting-edge technologies, and drive cost optimization. Additionally, you will build Kubernetes-native storage operators and self-service platforms, optimize end-to-end data paths, and implement monitoring and alerting systems. This position offers competitive health insurance, flexible time off, and a 401(k) plan.
- Design and deliver multi-petabyte storage systems for AI workloads, architect high-performance parallel filesystems and object stores.
- Build Kubernetes-native storage operators and self-service platforms for automated provisioning and performance isolation.
- Optimize end-to-end data paths for high throughput, design multi-tier caching architectures, and implement intelligent prefetching.
- Storage benchmarking and profiling tools (fio, iperf3, iostat, blktrace)
- Linux Storage Stack: Advanced knowledge of filesystems (ext4, xfs), LVM, NVMe optimization, RAID configurations
- Kubernetes Storage: CSI drivers, StatefulSets, PersistentVolumes, storage operators, and custom controllers
- History of technical leadership: designing systems that significantly improved performance (>3x), reliability (99.9%+ uptime), or cost efficiency
- 8+ years in storage engineering with 3+ years managing distributed storage at multi-petabyte scale
- Storage optimization for GPU workloads, RDMA/InfiniBand networking, parallel filesystem optimization (100+ GB/s aggregate cluster throughput)
- Object Storage: Production experience with S3, MinIO, Ceph, or R2 including performance optimization and cost management
- Infrastructure as Code: Terraform, Ansible, Helm, GitOps (ArgoCD)
- Strong coding skills in Go and Python with demonstrated ability to build production-grade tools
- BS/MS in Computer Science, Engineering, or equivalent practical experience
- Proven track record deploying and operating high-performance storage for GPU/HPC clusters
- Distributed Storage Systems: Deep expertise in WekaFS, Lustre, GPFS, BeeGFS, or similar parallel filesystems at multi-petabyte scale
- Programming: Go and Python for automation, operators, and tooling
- Deep Kubernetes and cloud-native storage experience in production environments
- Observability: Prometheus, Grafana, Thanos architecture and operations
- GPU Direct Storage (GDS), NVMe-oF, storage networking (100GbE/400GbE)
- ML/AI storage patterns (model weights, checkpointing, dataset caching)
- Kubernetes operator development (controller-runtime, kubebuilder)
- Storage snapshots, cloning, and thin provisioning
- Backup and disaster recovery (Velero, Restic, cross-region replication)
- Storage encryption (at-rest and in-transit), security and compliance
Similar roles you might like
See all →This is an external listing. JobSpring does not represent or verify the employer. Report this listing