Develop mid- and post-training safety methods for FLUX models, including safety SFT, preference optimization, RL with safety rewards, and concept erasure.
Build safety data filtering pipelines and classifiers that operate at pretraining scale, screening billions of images, video clips, and captions with optimized precision/recall trade-offs.
Ensure safety robustness for open-weight releases, addressing adversarial prompts and downstream fine-tuning.
Extend safety measures to the text components such as prompts, captions, and text encoders conditioning the models.
Design evaluations and red-teaming processes that balance harmful generation reduction with image quality and prompt following.
Collaborate directly with pretraining and post-training teams to integrate safety methods into release models and contribute to relevant publications.
Requirements
Hands-on research experience in visual safety related to text-to-image, text-to-video, or vision-language models including harm classifiers, safe generation methods, or safety benchmarks.
Proven expertise in at least one area:
Large-scale safety data filtering with classifiers or filtering pipelines over hundreds of millions of samples.
Safety mid- or post-training techniques such as safety SFT, DPO, RL with safety objectives, or unlearning in large generative models.
Deep familiarity with concept erasure methods (e.g., ESD, UCE, adversarial unlearning) and understanding of their failure modes like concept recovery and collateral damage.
Track record of shipping work demonstrated by top-venue publications (NeurIPS, ICML, ICLR, CVPR, ICCV, ECCV) paired with production impact or clear production achievements at a frontier lab.
Experience with distributed training at multi-node scale.
Strong PyTorch fluency and ability to write maintainable research code; effective use of coding agents to accelerate work with thorough validation.
Experience managing multi-agent workflows by allocating tasks appropriately to improve speed and maintain high quality.
Nice to haves
Knowledge of maintaining safety after release despite downstream fine-tuning of open weights.
Experience with safety challenges in video generation, addressing harms emerging across frames.
Background in audio safety, especially for jointly generated audio and video modalities.
Experience unlearning concepts from text encoders or building safety-tuned encoders like Safe-CLIP.
Work on text safety including LLM safety tuning, prompt and jailbreak defenses, or text safety classifiers across multiple languages.
Applied latent-space safety methods such as activation steering or safe guidance.
Built vision-language model (VLM) based safety judges or guard models for filtering or evaluation.
Perks and benefits
Competitive compensation ranging from EUR 160K to 220K plus equity, depending on profile.
Hybrid working model with required presence in Freiburg at least 2 days per week or one full week every other week, with travel costs covered.
Work in a distributed team that values in-person collaboration and balances flexibility with accessibility.
Culture grounded in being obsessed with research excellence, maintaining low ego, boldness in shipping work, and kindness among team members.