Member of Technical Staff - Safety

What you'll be doing

  • Develop mid- and post-training safety methods for FLUX models, including safety SFT, preference optimization, RL with safety rewards, and concept erasure.
  • Build safety data filtering pipelines and classifiers that operate at pretraining scale, screening billions of images, video clips, and captions with optimized precision/recall trade-offs.
  • Ensure safety robustness for open-weight releases, addressing adversarial prompts and downstream fine-tuning.
  • Extend safety measures to the text components such as prompts, captions, and text encoders conditioning the models.
  • Design evaluations and red-teaming processes that balance harmful generation reduction with image quality and prompt following.
  • Collaborate directly with pretraining and post-training teams to integrate safety methods into release models and contribute to relevant publications.

Requirements

  • Hands-on research experience in visual safety related to text-to-image, text-to-video, or vision-language models including harm classifiers, safe generation methods, or safety benchmarks.
  • Proven expertise in at least one area:
    • Large-scale safety data filtering with classifiers or filtering pipelines over hundreds of millions of samples.
    • Safety mid- or post-training techniques such as safety SFT, DPO, RL with safety objectives, or unlearning in large generative models.
  • Deep familiarity with concept erasure methods (e.g., ESD, UCE, adversarial unlearning) and understanding of their failure modes like concept recovery and collateral damage.
  • Track record of shipping work demonstrated by top-venue publications (NeurIPS, ICML, ICLR, CVPR, ICCV, ECCV) paired with production impact or clear production achievements at a frontier lab.
  • Experience with distributed training at multi-node scale.
  • Strong PyTorch fluency and ability to write maintainable research code; effective use of coding agents to accelerate work with thorough validation.
  • Experience managing multi-agent workflows by allocating tasks appropriately to improve speed and maintain high quality.

Nice to haves

  • Knowledge of maintaining safety after release despite downstream fine-tuning of open weights.
  • Experience with safety challenges in video generation, addressing harms emerging across frames.
  • Background in audio safety, especially for jointly generated audio and video modalities.
  • Experience unlearning concepts from text encoders or building safety-tuned encoders like Safe-CLIP.
  • Work on text safety including LLM safety tuning, prompt and jailbreak defenses, or text safety classifiers across multiple languages.
  • Applied latent-space safety methods such as activation steering or safe guidance.
  • Built vision-language model (VLM) based safety judges or guard models for filtering or evaluation.

Perks and benefits

  • Competitive compensation ranging from EUR 160K to 220K plus equity, depending on profile.
  • Hybrid working model with required presence in Freiburg at least 2 days per week or one full week every other week, with travel costs covered.
  • Work in a distributed team that values in-person collaboration and balances flexibility with accessibility.
  • Culture grounded in being obsessed with research excellence, maintaining low ego, boldness in shipping work, and kindness among team members.
AI Summary ✨

Similar jobs

  • Applied AI Engineer (InfoSec)

    Revolut9 locationsa day ago
    NewRemoteRegistered sponsor
    ≈ €9,700 – €12,000 / month base
  • Member of Technical Staff, Search

    CohereLondon, UK2 days ago
    Remote EMEARegistered sponsor
  • See all jobs in Germany →