Associate Distinguished Engineer - Solution Architect

Key Responsibilities

• Define and maintain the overall technical architecture for the KM data platform, spanning ingestion, storage (Databricks/Unity Catalog), transformation, security, and consumption layers.
• Translate multi-year program roadmaps into phased architectural plans, balancing near-term pilot needs against long-term scalability.
• Set architectural standards and guardrails for data modeling, pipeline design, and catalog registration, in partnership with the Senior Data Modeler and Data Engineers.
• Evaluate and recommend platform components, tools, and integration patterns (e.g., Unity Catalog vs. alternative catalogs, orchestration tools, vector search/embedding infrastructure).
• Lead technical design reviews and ensure alignment across data engineering, security/privacy, and downstream product teams (Knowledge Products, Research Products).
• Own non-functional requirements: scalability, performance, security, cost, and reliability of the platform architecture.
• Serve as the primary technical point of contact for architecture workshops and planning sessions (e.g., multi-day cross-functional architecture planning events).
• Assess technical risk and dependencies across workstreams and flag issues to the Engineering Manager and program leadership.
• Stay current on Databricks platform capabilities and enterprise AI/knowledge management architecture trends, bringing recommendations back to the team.

Required Qualifications 

• 8+ years of experience in data/solution architecture roles, with significant hands-on or architectural experience in Databricks/Lakehouse environments.
• Deep understanding of Unity Catalog governance, data product design, and security classification models.
• Demonstrated experience architecting platforms that handle both structured and unstructured content at scale.
• Experience architecting for AI/LLM-driven consumption patterns (retrieval-augmented generation, vector search, agent-based data access) is highly valued.
• Strong track record of translating business/program roadmaps into technical architecture and staged delivery plans.
• Excellent stakeholder management skills — able to work across engineering, product, legal/privacy, and executive audiences.
• Experience leading architecture reviews and setting technical standards for a growing engineering team.

Preferred Qualifications

• Experience in professional services, consulting, or other knowledge-intensive industries.
• Familiarity with enterprise search/knowledge platforms (Glean, SharePoint, ServiceNow) and their integration patterns.
• Experience navigating legal/risk/privacy review processes for data platforms.
• Prior experience standing up a data platform team from a nascent or pilot stage to a scaled production capability. Success Metrics (First 6–12 Months)
• Documented target-state architecture for the KM data platform, validated with key stakeholders (Engineering Manager, Data Modeler, Product Owners).
• Architectural standards adopted across data modeling and engineering workstreams.
• Clear, staged technical roadmap aligned to the broader program timeline (e.g., through 2027). 

Must have skills: Azure Data Factory, Data Modeling (Strong), Databricks