Requirements:

• Experience implementing, training, testing and evaluating performance of Machine Learning systems;
• Strong Python knowledge. A big plus for proven familiarity and experience with OOP principles;
• Knowledge and experience developing and evaluating GenAI / LLM solutions, including automated eval harnesses and prompt engineering;
• Experience with statistical analysis, experiment design, and failure mode analysis in LLM systems;
• A strong product mindset with the ability to work independently in a cross-functional and cross-team environment;
• Good communication skills and ability to get the point across to non-technical individuals and SMEs;
• Strong problem solving skills with the ability to help refine problem statements and figure out how to solve them.

Nice to Haves:

• Familiarity with automating operational processes via technical solutions, for example Large Language Models
• Experience implementing fine-tuning, reinforced learning alignment and evaluation techniques within an LLM training pipeline.
• Familiarity with agentic frameworks such as LangGraph or similar.
• Willingness to get hands dirty reading many, many historical operational cases.
• Working knowledge of Java.

What You'll Be Doing:

• End-to-End Automation & EDD LLMs
• Evaluation Framework & Labeling Taxonomy
• Prompt Optimization & Experimentation
• Shadow Testing & Monitoring
• Full-Stack Deployment
• Human-in-the-Loop Architecture
• Strategic Demand Deflection
• Technical Leadership & Mentorship

Perks and Benefits:

• People without borders - without judgement or prejudice
• Qualifications aren't very important, experience and articulation are valued
• Encouragement for diverse team members, especially from under-represented demographics