Sr. Software Engineer, Inference

About Anthropic

• Reliable, interpretable, and steerable AI systems
• Safe and beneficial AI for users and society
• Committed team of researchers, engineers, policy experts, and business leaders

About the Role:

• Building and maintaining critical systems serving millions of users worldwide
• Maximizing compute efficiency and enabling breakthrough research

Strong candidates may also have experience with:

• High-performance, large-scale distributed systems
• Implementing and deploying machine learning systems at scale
• Load balancing, request routing, or traffic management systems
• LLM inference optimization, batching, and caching strategies
• Kubernetes and cloud infrastructure (AWS, GCP)
• Python or Rust

You may be a good fit if you:

• Have significant software engineering experience, particularly with distributed systems
• Are results-oriented, with a bias towards flexibility and impact
• Pick up slack, even if it goes outside your job description
• Want to learn more about machine learning systems and infrastructure
• Thrive in environments where technical excellence directly drives both business results and research breakthroughs
• Care about the societal impacts of your work

Representative Projects:

• Designing intelligent routing algorithms optimizing request distribution
• Autoscaling compute fleet to match supply with demand
• Building production-grade deployment pipelines for releasing new models
• Integrating new AI accelerator platforms for hardware-agnostic advantage
• Contributing to new inference features and supporting new model architectures
• Managing multi-region deployments and geographic routing

Perks and Benefits

• Competitive compensation and benefits
• Optional equity donation matching
• Generous vacation and parental leave
• Flexible working hours
• Lovely office space in San Francisco for collaboration