Requirements

• Bachelor's degree or equivalent practical experience.
• 8 years of experience working with Large Language Models (LLMs) and building agents.

Nice to haves

• Solid understanding of essential concepts such as tokens, context, Retrieval-Augmented Generation (RAG), and function calling.
• Solid understanding of core ML concepts, training algorithms, and best practices for evaluation.
• Strong understanding of data and familiarity with basic statistical analysis concepts (e.g., variance, p-value, bias) and general data science principles.
• Familiarity with running quality interactions, including understanding business goals, defining technical metrics, implementing evaluation frameworks, designing experiments, analyzing results, performing RCAs, formulating hypotheses, and conducting ablation or live experiments.
• Proficiency in Python.

What you'll be doing

• Execute quality iterations on reference agents identifying potential model or agent framework improvements.
• Source or facilitate creation of datasets for evaluation and training to improve model performance for Cloud customers.
• Design new agentic systems and context engineering algorithms, utilizing existing models and identifying the need for new training/evaluation objectives where necessary.
• Define and implement metrics that correspond to business problems.
• Prototype and iterate on solutions working closely with customers, product management, and business development.