5+ years experience designing, building, and operating high-throughput, low-latency distributed production systems with strict availability requirements
Strong programming skills in Go, Python, or TypeScript with experience writing clean, concurrent, maintainable, and well-tested backend services
Extensive production experience with Kubernetes (custom controllers/operators, scheduling, networking, CNI/CSI) and multi-cloud infrastructure (AWS, GCP, or bare-metal environments)
Proven track record of re-architecting live systems, executing large-scale data or traffic migrations, and maintaining strict API backwards compatibility under load
Hands-on experience operating tier-1 production services including observability, capacity planning, staged rollouts, and incident response
Ability to write concise architectural proposals and communicate complex technical tradeoffs clearly across distributed engineering teams
Nice to Haves
Experience building or operating GPU infrastructure, ML inference engines (e.g., Triton, vLLM, Cog), or container image distribution/caching systems optimized for fast cold starts
Experience with Cloudflare Workers, Workers AI, or edge/serverless compute platforms
Familiarity with high-performance networking, load balancing, and multi-region traffic routing
Experience working on developer-facing APIs, usage-based billing/metering pipelines, or multi-tenant sandboxing and workload isolation
What You'll Be Doing
Design, implement, and maintain backend services powering Replicate's inference lifecycle including public APIs, request queueing, workload scheduling, container orchestration, and usage metering/billing
Architect and execute migration of Replicate models and inference traffic to Workers AI with zero customer disruption
Profile and improve system throughput, cold-start latency, and GPU utilization across multi-cloud Kubernetes clusters and Cloudflare's edge infrastructure
Participate in on-call rotation, lead triage and mitigation during production incidents, author post-incident reviews, and build automated remediation and observability tools
Author technical design documents, conduct code and architecture reviews, and mentor engineers on distributed systems best practices and operational excellence
Collaborate with Workers AI, networking, storage, and security teams to integrate Replicate’s platform into Cloudflare’s broader AI stack
Perks and Benefits
Opportunity to work at a highly ambitious, large-scale technology company with a strong mission to protect a free and open Internet
Engagement with impactful projects like Project Galileo, the Athenian Project, and the development of privacy-centric services such as 1.1.1.1
Commitment to equal employment opportunity, diversity, inclusiveness, and providing reasonable accommodations for applicants with disabilities
Chance to contribute to innovative AI and cloud infrastructure solutions within a collaborative and supportive culture