Principal Software Architect (Data Platform)

Join SecurityScorecard as a Principal Software Architect to lead the design of our data platform. This individual contributor role will report to the Chief Architect and work closely with engineering leadership, Product, and Data Science. You will own the system design for our data platform, define service boundaries and data contracts, design the lakehouse, architect the analytical serving layer, and engineer data quality and observability into the platform. You should have 10+ years of software or data engineering experience, including significant time architecting large-scale data platforms.

• Lead the system design of the data platform, ensuring the architecture supports the ingestion, processing, storage, and serving of large-scale measurement data.
• Define service boundaries and data contracts between producers and consumers, including schema ownership, compatibility rules, and handling breaking changes.
• Engineer data quality and observability into the platform, designing for correctness and reproducibility in the ratings pipeline, including backfills and historical restatement.

• Strong Python and PySpark, solid Java for Flink stream processing, and enough Scala to read and reason about an existing Spark codebase
• Experience owning a large-scale data migration, including preserving history and correctness through a cutover
• Experience architecting OLAP and analytical serving layers (ClickHouse, Druid, Pinot, BigQuery, Snowflake or similar)
• Comfort operating as a senior individual contributor, driving outcomes through prototyping and technical credibility
• 10+ years of software or data engineering experience, including significant time architecting large-scale data platforms
• Hands-on experience designing lakehouse storage in production: columnar formats such as Parquet, open table formats such as Iceberg, and the partitioning, compaction, and schema evolution decisions that come with them
• A track record of influence without authority: presenting a technical direction to skeptical engineers and earning genuine buy-in, and giving rigorous design review feedback on systems you didn't build yourself
• Strong technical writing and mentorship: TDRs, design docs, and decision records that teams can act on without hand-holding, plus a history of raising the technical bar around you
• A track record of building data quality, contracts, and observability as engineered system properties, meaning assertions, schema enforcement, and lineage that live in code
• Deep expertise in stream and batch processing at scale with Kafka, Flink, and Spark or close equivalents, and clear judgment about which path a given workload belongs in
• Strong distributed systems fundamentals as they apply to data: exactly-once versus at-least-once semantics, ordering, backpressure, late and out-of-order data, and pipeline failure modes
• Experience building the data layer underneath ML or LLM systems: feature stores, vector stores, or retrieval pipelines
• Experience with internet-scale scan, telemetry, or observability data, or a cybersecurity industry background
• Track record of bringing platform cost down at scale through storage tiering, query governance, or compute right-sizing