Requirements

• Proven experience in systems engineering, with a primary focus on distributed, high-performance systems.
• Expert proficiency in Rust programming, particularly in an asynchronous environment.
• Deep understanding and hands-on experience with relevant networking and application protocols (e.g., TCP, HTTP, WebSocket).
• Solid experience with scaling and performance optimization techniques, including load balancing and caching in a distributed environment.

Nice to Haves

• Demonstrable experience with container orchestration platforms, specifically Kubernetes and/or Nomad.
• Familiarity with the unique architectural challenges involved in large-scale inference serving (e.g., LLMs and diffusion models).

What You'll Be Doing

• Platform Architecture: Develop and maintain core components of the serverless inference platform to ensure high availability and scalability for Cloudflare users.
• Optimization & Performance: Optimize the model scheduling system to significantly increase efficiency and resource utilization across our inference infrastructure, and implement improvements to request routing logic to reduce latency for end-users.
• System Reliability: Drive significant, measurable improvements in the platform's reliability and resilience by identifying and mitigating systemic risks.
• Observability: Expand and refine the observability stack (metrics, logging, tracing) and fine-tune alerts to proactively identify and resolve production issues.
• Technical Leadership: Lead complex, cross-functional technical projects from initial concept and design through final deployment and operationalization.
• Mentorship: Act as a mentor to junior engineers and actively contribute to cultivating a strong, collaborative engineering culture within the team.

Perks and Benefits

Cloudflare provides reasonable accommodations to qualified individuals with disabilities. Please tell us if you require a reasonable accommodation to apply for a job. Examples of reasonable accommodations include, but are not limited to, changing the application process, providing documents in an alternate format, using a sign language interpreter, or using specialized equipment.
If you require a reasonable accommodation to apply for a job, please contact us via e-mail at hr@cloudflare.com or via mail at 101 Townsend St. San Francisco, CA 94107.