Requirements

Hands-on cybersecurity expertise in at least one area: malware analysis and reverse engineering, incident response and threat hunting, detection engineering, SOC operations
Demonstrated experience building evaluations, benchmarks, or datasets for AI or LLM systems with shareable artifacts
Strong Python and data tooling skills with production-quality evaluation code and ability to review others' code
Practical experience with LLMs and agentic systems including tool use, multi-step planning, sandboxing, and instrumentation/tracing of agent behavior
Minimum 2 years of technical leadership experience with a record of growing researchers and engineers while remaining hands-on
Ability to define, design, and standardize evaluation methodologies and reproducible testing pipelines, and defend them under scrutiny
Broad knowledge of cybersecurity landscape: attack techniques, defensive controls, and analyst workflows augmented by AI
Exceptional written and spoken communication skills to present findings and limitations to executives and researchers
Proven experience using AI technologies to improve decision-making, workflows, efficiency, and business outcomes

Nice to Haves

Published research or talks on AI evaluation, LLM judging, or AI security
Experience with adversarial testing or red teaming of AI systems and scoring agents based on observed side effects
Experience building LLM judges and measuring agreement with human experts
Experience with SOC platforms, SIEM/SOAR tooling, and detection engineering
Understanding of MITRE ATT&CK and its relation to detection and response workflows
Relevant security certifications such as GREM, GCFA, GCIH, GCIA, OSCP or equivalent
Experience fine-tuning or distilling models on expert-labeled decision data

Design and build benchmark datasets for cybersecurity agent tasks including task definition, expert labeling, scoring rubrics, and release
Build and calibrate LLM-based and programmatic judges against expert labels, measure agreement, and publish disagreements
Create reproducible evaluation harnesses that capture agent actions and traces for defensible and comparable results
Run and publish ongoing scorecards of CrowdStrike agents and frontier models covering protection, correctness, and usefulness
Investigate and analyze failure modes of AI agents including causes and supporting evidence
Write internal research reports and external papers or talks establishing benchmark credibility
Lead, mentor, and grow a team of researchers and engineers; review their designs, labels, and code
Set strategy, roadmap, and success metrics for AI and agentic system evaluations across security use cases
Define and enforce evaluation methodologies and shared formats used across benchmarking and red teaming
Coordinate subject matter expert labeling with SOC, research, and malware teams ensuring quality standards
Collaborate with engineering, product, and research to translate evaluation findings into pass/fail criteria and improvements
Communicate results clearly to both technical and executive audiences, including what was not tested

Market leader in compensation and equity awards
Comprehensive physical and mental wellness programs
Competitive vacation and holidays for recharge
Paid parental and adoption leaves
Professional development opportunities for all employees regardless of level or role
Employee networks, geographic neighborhood groups, and volunteer opportunities to build connections
Vibrant office culture with world-class amenities
Great Place to Work Certified™ globally