Software Engineer – Annotation & Evaluation (AI Platform)

Requirements:

• You have 3+ years of experience as a backend or platform engineer, ideally with exposure to ML systems or data engineering.
• You’re comfortable working in Python, Go, or similar languages and have built or maintained backend services or pipelines.
• You’re excited by the challenge of evaluating ML/LLM models in real-world contexts.
• You understand the importance of data quality, safety checks, and explainability in AI systems.

What You'll Be Doing:

• Contribute to evaluation infrastructure that enables model comparison, performance tracking, and trust & safety scoring.
• Build services and pipelines to support annotation workflows, benchmarking dashboards, and result aggregation.
• Collaborate with scientists and engineers to build and scale tools for evaluating models in controlled and production-like environments.
• Help maintain integrations with data labeling platforms and model evaluation libraries.
• Write clean, testable code to ensure reliability of evaluation tooling and data pipelines.

Perks and Benefits:

• New hire stock equity (RSUs) and employee stock purchase plan (ESPP)
• Continuous professional development, product training, and career pathing
• Intradepartmental mentor and buddy program for in-house networking
• An inclusive company culture, ability to join our Community Guilds (Datadog employee resource groups)
• Access to Inclusion Talks, our internal panel discussions
• Free, global mental health benefits for employees and dependents age 6+
• Competitive global benefits