What You'll Be Doing

• Design, build, and operate distributed training and inference infrastructure for large language models, including Mixture-of-Experts models
• Scale training with parallelism strategies (data, tensor, pipeline, and expert), memory optimization, checkpointing, and failure recovery
• Build inference and rollout systems that support evaluation, experimentation, and reinforcement learning without slowing down training
• Build execution environments and sandboxes for online reinforcement learning and agent post-training
• Own continuous performance validation with regular benchmarks and dashboards to detect regressions
• Improve the researcher experience through job submission, scheduling, quotas, GPU utilization, and observability enhancements
• Collaborate with platform teams on cluster capacity, networking, and scheduling across GPU providers
• Investigate performance and reliability problems across hardware, software, and distributed systems

Requirements

• Depth in distributed systems and machine learning infrastructure for training and inference at scale
• Experience with frameworks like Ray, Kubernetes, Slurm, or similar is a plus
• Proficiency in Python and at least one systems programming language such as Rust, C++, or Go
• Experience operating large-scale training or serving infrastructure, or schedulers and orchestration systems for GPU or high-performance workloads
• Understanding of throughput and latency limits in distributed inference workloads including communication, memory, scheduling, and straggler issues
• Strong debugging, profiling, and incident diagnosis skills, ideally including on-call experience
• Ability to explain design and performance trade-offs clearly to technical and non-technical audiences
• Prior experience training machine learning models is valuable but not essential for candidates with strong distributed systems or HPC backgrounds

Nice to Haves

• Familiarity with distributed training frameworks such as Megatron-LM, TorchTitan, or DeepSpeed
• Experience with large language model inference engines such as vLLM or SGLang, including batching and KV-cache management
• Experience with reinforcement learning training systems and rollout infrastructure
• Knowledge of collective communication (NCCL), RDMA, or low-latency networking
• Experience with GPU benchmarking, profiling, and cluster performance monitoring
• Experience building sandboxed code execution or large-scale evaluation infrastructure
• Contributions to open-source machine learning or distributed systems projects

Perks and Benefits

• Competitive global benefits
• New hire stock equity (RSUs) and employee stock purchase plan (ESPP)
• Opportunities to collaborate with colleagues across Datadog offices in New York City and Paris
• Opportunities to attend and present at conferences and meetups
• Intra-departmental mentor and buddy program for in-house networking
• Inclusive company culture with access to employee resource groups known as Community Guilds
• Benefits and growth opportunities may vary based on country and nature of employment