Requirements

You have hands-on experience with retrieval-augmented systems, search, ranking, or knowledge-based AI applications, and understand what drives high-quality grounded responses.
You have worked on knowledge-heavy or policy-constrained domains where correctness, consistency, and trust matter.
You have experience building feedback loops from user behavior, QA signals, annotations, or operational outcomes into system improvements.
You are able to operate at both strategic and hands-on levels: defining long-term direction while also diving into data, experiments, and failure analysis.
You have strong communication and stakeholder management skills, and can align engineering, product, and operations around quality strategy and trade-offs.
Experience with LLM-based pipelines or agentic AI systems is a strong plus, especially in areas such as tool use, routing, memory, multi-step reasoning, or human-AI orchestration.

What You'll Be Doing

Lead alignment of Customer Support AI system behavior to Bolt’s business goals, customer experience standards, and policy requirements, improving answer quality, consistency, grounding, and escalation behavior.
Build and scale knowledge mining capabilities that turn conversations, support tickets, workflows, policies, and operational data into reusable knowledge assets and improvement signals.
Own the evaluation strategy for LLM-powered customer support systems, including offline benchmarks, golden datasets, human review workflows, online experimentation, and production quality metrics.
Develop feedback loops that connect production failures and human corrections back into prompts, retrieval, benchmarks, and model behavior, enabling continuous improvement of automation quality.
Partner closely with engineers working on AI application infrastructure, contributing ML expertise to areas such as tool-use quality, routing behavior, memory quality, and multi-step task performance.
Influence roadmap and technical direction across teams by bringing clarity to trade-offs in model behavior, retrieval quality, experimentation, and business impact.

Play a direct role in shaping the future of mobility.
Impact millions of customers and partners in 600+ cities across 50+ countries.
Work in fast-moving autonomous teams with some of the smartest people in the world.
Accelerate your professional growth with unique career opportunities.
Get a rewarding salary and stock option package that lets you focus on doing your best work.
Enjoy the flexibility of working in a hybrid mode with a minimum of 2 days in the office each week to foster strong connections and teamwork.
Take care of your physical and mental health with our wellness perks.