What You'll Be Doing

AI-driven version migration automation for Spark and JVM
Building agentic tooling to analyze compatibility, generate PRs, and orchestrate multi-datacenter testing
Developing engine-agnostic batch execution
Ensuring platform reliability and scalability
Implementing Spark Pod Autosizing — CPU and memory allocations
Understanding tradeoffs across execution models

Requirements

6+ years of experience
Significant backend programming experience in one or more languages: Go, Java, Rust, Python, Scala
Knowledge of Spark internals and infrastructure — executor lifecycle, memory model, dynamic allocation, resource tuning
Experience building platforms or infrastructure that other engineers depend on
Background as a distributed systems engineer building high performance large scale systems
Familiarity with Kubernetes fundamentals
Excellent English communication skills
High proactivity and experience collaborating across teams
Excitement for leveraging AI tools to enhance coding, problem solving, and building—or eagerness to learn
Demonstrated ability to use AI coding tools in day-to-day workflows and validate, critique, and refine AI-generated output

New hire stock equity (RSUs) and employee stock purchase plan (ESPP)
Continuous professional development, product training, and career pathing
Intradepartmental mentor and buddy program for in-house networking
Inclusive company culture with the ability to join Community Guilds (employee resource groups)
Access to Inclusion Talks, internal panel discussions
Free, global mental health benefits for employees and dependents age 6+
Competitive global benefits