Requirements

Bachelor’s degree in computer science or a related STEM field
Experience programming AI accelerators (e.g. GPUs, custom silicon etc.) using AI frameworks such as PyTorch or similar
Experience developing custom kernels and compiler infrastructure to improve performance using low-level programming models such as CUDA, OpenCL or similar
Minimum 6+ years of experience developing and optimizing performance in modern C/C++
Must obtain work authorization in the country of employment at the time of hire, and maintain ongoing work authorization during employment

Nice to Haves

Experience with training and validating large-scale AI models, including parallelising models across several accelerators
Understanding of multiprocessing, including race conditions and communications between processes
Experience of evaluating model performance, e.g., with profilers and tuning hyperparameters
Thorough understanding of model and data parallelisms such as FSDP, tensor parallelism, model parallelism, expert parallelism, etc
Demonstrated experience of the model life cycle from pre-training and post-training to inference, dataset splits and shuffling, metrics, especially for large language models
Experience of developing, optimizing and validating kernels on GPUs or other accelerators

Work cross-functionally to co-design models to maximize pre-training and inference efficiency
Applying and driving state-of-the-art optimization techniques to our latest large-scale AI workloads running on Meta’s fleet of accelerators including functional development and maintenance
Profiling, analyzing, debugging, and optimizing large-scale workloads on our next-generation training superclusters
Optimization of the underlying processes of the whole vertical stack, from kernels, framework, communication, and firmware to layers and hyperparameters
Set direction and goals for the team related to project impact, capacity, and developer efficiency
Lead large and complex technical efforts across many engineers and teams from zero to one

Meta builds technologies that help people connect, find communities, and grow businesses
Move beyond 2D screens toward immersive experiences like augmented and virtual reality
Shape a future that will take us beyond what digital connection makes possible today
Equal Employment Opportunity
Opportunity for reasonable accommodations for qualified individuals with disabilities and disabled veterans