Requirements:

• Maintain large-scale pipelines for processing web corpora.
• Work on filtering and quality-scoring systems to identify high-value web documents.
• Analyze web data composition across domains, languages, and time periods.
• Develop and maintain highly-performant deduplication pipelines.
• Collaborate with cross-functional teams, including researchers and engineers, to ensure data pipelines meet the demands of cutting-edge language models.
• Strong software engineering skills, with proficiency in Python and experience building data pipelines.
• Familiarity with data processing frameworks such as Apache Spark, Apache Beam, Pandas, or similar tools.
• Experience working with large-scale web datasets.
• Knowledge of data quality assessment techniques and experimentation with data mixtures.
• A passion for bridging research and engineering to solve complex data-related challenges in AI model training.

Nice to Haves:

• Bonus: paper at top-tier venues (such as NeurIPS, ICML, ICLR, AIStats, MLSys, JMLR, AAAI, Nature, COLING, ACL, EMNLP).

What You'll Be Doing:

• Building and maintaining large-scale web data pipelines.
• Working on filtering and quality-scoring systems for web documents.
• Analyzing web data composition and quality across various criteria.
• Developing and enhancing deduplication pipelines.
• Collaborating with diverse teams to optimize data pipelines for advanced language models.

Perks and Benefits:

• A weekly lunch stipend of $75/£75 or equivalent in local currency.
• Full health and dental benefits, including mental health budget.
• RRSP matching, 401K, Pension Scheme.
• 100% Parental Leave top-up for up to 6 months.
• Annual enrichment benefits for arts & culture, fitness/wellness, quality time, and workspace improvement.
• Education & learning stipend for conferences, courses, and coaching.
• 6 weeks of paid vacation.
• Budget for traveling to other offices and annual company offsite.