Requirements

• Bachelor's degree or equivalent practical experience.
• 8 years of experience programming in C++, Java, Python, Kotlin, or Go.
• 5 years of experience testing and launching software products.
• 3 years of experience with software design and architecture.
• Experience integrating generative AI tools or LLM interfaces into workflows.

Nice to haves

• Master’s degree or PhD in Engineering, Computer Science, or a related technical field.
• 8 years of experience with data structures/algorithms.
• 3 years of experience in a technical leadership role leading project teams and setting technical direction.
• 3 years of experience working in a complex, matrixed organization involving cross-functional or cross-business projects.

What you'll be doing

• Research and implement novel techniques for accelerated evaluation creation, including automated task synthesis, programmatic verifiers, and model-assisted rubric induction.
• Architect pipelines to mine execution logs, agent traces, and runtime artifacts into high-fidelity, future-proof evaluations and training data that continuously self-improve.
• Design and build robust benchmarks evaluating complex, long-horizon agentic reasoning across code, math, and specialized domains; optimize for climbability, discriminability, and zero contamination.
• Build and scale platform software that automates quality gates such as AutoRaters and automated loss-pattern root-cause analyzers.
• Provide technical direction and architecture across AI Data and partner teams; align distributed engineering teams and establish flexible, reusable approaches for all benchmark types.

Perks and benefits

• Opportunity to work on next-generation technologies impacting billions of users worldwide.
• Collaboration across innovative projects in AI, machine learning, large-scale system design, and distributed computing.
• Supportive environment encouraging leadership, versatility, and tackling new technical challenges.
• Part of Google Cloud’s mission to accelerate digital transformation globally.
• Commitment to an inclusive workplace with equal employment opportunity.