Principal AI Engineer (Context, Agents and Context)

Join Elastic as a Principal AI Engineer, where you will lead the improvement loop for the Context Engine, a key component of our AI-driven products. You will work at the intersection of engineering, data science, and product, writing production code, designing evaluation and telemetry, and setting the technical bar for agentic behavior. This hybrid role offers flexible location and schedule, fully paid health coverage, generous vacation days, and more.

• Ownership of the production improvement loop for the Context Engine, including understanding and fixing failure modes.
• Defining and implementing safe iteration processes for agents and skills, including versioning, rollout, and regression coverage.
• Designing and implementing telemetry systems to capture relevant data from agent traces, tool calls, and knowledge retrieval.

• A track record of eval-driven product improvement: you have diagnosed agent or LLM behaviour from traces and user feedback, designed the evaluation that exposed the problem, shipped the fix and measured the outcome
• A pragmatic, low-ego style suited to a distributed, async-first team
• Experience running product experiments end to end: instrumentation, unattended execution, and interpreting results
• Direct experience building agents with state and memory, and iterating on prompts, skills and tool behaviour safely in production
• Comfort working with data scientists, engineers, and product managers as peers, translating between measurement and shipping, and communicating trade-offs clearly
• 10+ years of software engineering experience, with the recent years spent shipping and operating AI-driven products on real production traffic, ideally products with public APIs and data models that had to evolve without breaking customers
• Familiarity with MCP, including exposing public MCP servers and tools
• Strong backend engineering skills in either Python or TypeScript: APIs, stateful workflows, data pipelines and production services
• Experience designing telemetry for AI systems, and using it to make engineering and product decisions
• Experience building products with public APIs and evolving data models, and the judgement that comes with maintaining external contracts as a product changes
• Agentic retrieval experience, including knowledge representation
• Practical Elasticsearch experience
• Experience with agent frameworks such as LangGraph, CrewAI, Claude Agent SDK, or similar