Designing, building, and evolving scalable data pipeline and orchestration infrastructure for Wayve's next-generation autonomous driving data platform
Revamping the existing Harness system and developing a new DAG-based workflow platform
Building tooling and libraries to make data ingestion, processing, inference, and evaluation workflows simple, reliable, and consistent across teams
Ensuring the platform runs hundreds of parallel pipeline steps efficiently, with correct outputs, no conflicts, and effective resource prioritisation
Supporting batch inference and performance optimisation workflows for cost-effective and scalable evaluations
Improving platform reliability, quality, and usability, including incident response workflows aligned to business-defined SLOs
Requirements
Strong software engineering fundamentals with solid Python programming skills and experience designing, building, and maintaining production-grade systems
Experience building or operating DAG-based workflow orchestration and data pipeline systems using tools such as Airflow, Dagster, or Flyte
Experience with distributed computing frameworks such as Ray or Spark
Experience working with systems at scale, involving distributed systems, large data volumes, or high-throughput processing
Experience deploying and operating production workloads on Kubernetes, including troubleshooting and resource management
Familiarity with distributed event streaming platforms like Apache Kafka
Strong understanding of database principles and consistency models with practical familiarity with relational databases and data warehouses such as Postgres and ClickHouse
Strong collaboration skills to work closely with platform users and partner teams to translate requirements into scalable technical solutions
Nice to haves
Experience with machine learning infrastructure, inference systems, or ML performance optimisation
Exposure to large-scale batch processing, compute scheduling, or resource prioritisation
Experience building shared libraries, developer tooling, or internal platforms used across multiple teams
Perks and benefits
Salaries benchmarked against the market annually
Meaningful equity sharing in the ownership and long-term success of Wayve
Relocation support and visa sponsorship where applicable
Hybrid working model with core hours and access to vehicle workshops and labs
Learning and development budgets supporting training, conferences, and growth
Comprehensive benefits including health insurance, dental, enhanced maternity and paternity leave, retirement or pension schemes where applicable
Access to therapists, wellbeing partnerships, team socials, and more