Software Engineer (Inference AI/ML)

Join our Inference team as a Software Engineer (Inference AI/ML) and contribute to improving latency, reliability, and cost for model serving on our GPU platform. You will implement well-scoped features and fixes in Python/Go/C++, write tests and code comments, and participate in code reviews. You will also assist with basic metrics and dashboards, follow on-call runbooks, and contribute to performance experiments. This is a great opportunity for recent graduates or those with equivalent practical experience in computer science, electrical engineering, or related fields.

• Implement well-scoped features and fixes in Python/Go/C++ for model-serving services.
• Write tests, code comments, and short design docs; participate in code reviews.
• Contribute to performance experiments (e.g., request batching, concurrency, caching) with guidance.

• Foundations in data structures, algorithms, and networked services
• BS/MS in CS, EE, or related field, or equivalent practical experience
• Exposure to containers and Kubernetes (coursework or projects welcome)
• Experience with Python or Go (C++ a plus) and Linux fundamentals; Git/CI basics
• Curiosity about GPU inference concepts (micro-batching, KV cache, streaming)
• Coursework/research with PyTorch or TensorFlow; simple CUDA projects a plus
• Internship or project that deployed a microservice or ML inference demo
• Familiarity with Grafana/Prometheus/OpenTelemetry or similar tooling