Requirements

Bachelor’s degree in computer science or a related STEM field
Experience programming AI accelerators (e.g. GPUs, custom silicon etc.) using AI frameworks such as PyTorch or similar
Experience developing custom kernels and compiler infrastructure to improve performance using low-level programming models such as CUDA, OpenCL or similar
Minimum 5+ years of experience developing and optimizing performance in modern C/C++
Must obtain work authorization in the country of employment at the time of hire, and maintain ongoing work authorization during employment

Nice to Haves

Experience with training and validating large-scale AI models, including parallelizing models across several accelerators
Understanding of multiprocessing, including race conditions and communications between processes
Experience of evaluating model performance, e.g., with profilers and tuning hyperparameters
Thorough understanding of model and data parallelisms such as FSDP, tensor parallelism, model parallelism, expert parallelism, etc
Demonstrated experience of the model life cycle from pre-training and post-training to inference, dataset splits and shuffling, metrics, especially for large language models
Experience of developing, optimizing and validating kernels on GPUs or other accelerators

Work cross-functionally to co-design models to maximize pre-training and inference efficiency
Applying and driving state-of-the-art optimization techniques to our latest large-scale AI workloads running on Meta’s fleet of accelerators including functional development and maintenance
Profiling, analyzing, debugging, and optimizing large-scale workloads on our next-generation training superclusters
Optimization of the underlying processes of the whole vertical stack, from kernels, framework, communication, and firmware to layers and hyperparameters
Lead large and complex technical efforts across many engineers and teams from zero to one

Meta builds technologies that help people connect, find communities, and grow businesses
Opportunity to work on maximizing performance of AI models on GPUs or custom silicon
Be part of the AI Applications Engineering team focused on maximizing training and inference performance
Collaborate with partner teams to ensure end-to-end performance of large-scale pre-training and inference