Research Engineer (Agentic Behavior – Kotlin AI Value Stream)

Requirements:

• Hands-on experience building evaluation or analysis pipelines for LLMs or AI coding agents in a research or production setting.
• Strong Python engineering skills (at least three years), with the ability to write clean, maintainable code in data-heavy and ML-adjacent codebases.
• Experience with data analysis at scale: querying large datasets (SQL/Athena), building data pipelines, and performing statistical analysis of experimental results.
• The ability to own projects end to end – from identifying a problem in agent traces to designing an eval, running experiments, and shipping a fix.
• A product-aware mindset: You care about how agents are actually used by developers and can translate real failure modes into evaluation and training work.
• Familiarity with Kotlin or a strong willingness to develop deep Kotlin expertise (you'll be living in Kotlin codebases daily).

Nice to haves:

• Post-training LLMs: SFT, RLHF, DPO, GRPO – either hands-on training or designing the data and reward pipelines that feed into training.
• Modern deep learning frameworks (PyTorch) and LLM training stacks (TRL, verl, Megatron, or similar).
• AI agent development: tool-using agents, multi-step coding workflows, agentic frameworks.
• Evaluation frameworks and tools: Inspect AI, Promptfoo, LM-evaluation-harness, or custom eval pipelines.
• Experiment tracking and observability: Weights & Biases, MLflow, Langfuse, or similar.
• The Kotlin ecosystem: Android, Gradle, KMP, Spring, Ktor – with an understanding of the developer workflows that agents need to support.
• Contributing to or maintaining open-source projects, especially benchmarks or evaluation tools.

What you'll be doing:

• Build tools for agentic error analysis:
• Design and implement tooling to systematically capture, classify, and analyze errors that AI coding agents make when generating Kotlin code.
• Build observability pipelines over agentic traces – mining patterns from agent sessions in JetBrains IDEs, Junie, Claude Code, Cursor, and other coding agents.
• Build evaluation pipelines:
• Design, implement, and maintain evaluation pipelines that measure Kotlin code generation quality across dimensions.
• Build simulation environments where coding agents can be measured on realistic Kotlin developer tasks.
• Own evaluation infrastructure: metrics, experiment tracking, automated regression checks, and reproducible benchmarking.
• Research methods for improving agent and model behavior on Kotlin:
• Experiment with post-training techniques to improve how models handle Kotlin-specific patterns, idioms, and frameworks.
• Investigate context engineering approaches and run experiments to measure impact.
• Collaborate with model providers to translate Kotlin-specific findings into model improvements.
• Build public Kotlin benchmarks:
• Design and build open-source benchmarks that measure AI coding agent performance on Kotlin tasks.
• Create task datasets covering the breadth of Kotlin usage and maintain benchmarks as models improve.

Perks and Benefits:

• Strong base salary. We offer competitive pay that reflects your skills and experience.
• Flexible work location. Enjoy the freedom to work from home or from the office.
• Remote work. Spend up to 30 days per year working remotely from abroad.
• Extra time off. More days to relax, recharge, and do the things you love.
• Medical insurance allowance. Enjoy peace of mind for you and your family.
• Learning and development opportunities. Access to conferences, courses, and language classes.
• Relocation support. We help make your move as smooth and stress-free as possible.
• Language classes. Pick up the local language or sharpen your English skills.
• Fuel your day. Enjoy a hot meal or receive a lunch allowance on workdays.
• Mental health support. To help you feel your best, we provide easy access to professional mental health services.
• Sports benefit. Enjoy an on-site gym or sports club stipend.
• Internal events. Join company-wide celebrations and team gatherings.

*Some benefits may vary depending on location.