Manager, AI Benchmarking and Evaluation Research

Job not on LinkedIn

🔥 3 minutes ago

🇷🇴 Romania – Remote

⏰ Full Time

🟡 Mid-level

🟠 Senior

🤖 Artificial Intelligence

👻 Ghost score 10%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

CrowdStrike

WebsiteLinkedInAll Job Openings

5001 - 10000 employees

Founded 2011

🔒 Cybersecurity

☁️ SaaS

🤖 Artificial Intelligence

Cybersecurity • SaaS • Artificial Intelligence

CrowdStrike is a cybersecurity company that provides cloud-based security services to stop breaches. It is recognized as a leader in endpoint protection, identity and cloud security, and managed detection and response. CrowdStrike's platform, Falcon, integrates artificial intelligence to offer real-time visibility, detection, and protection against sophisticated cyber threats. The company is lauded for its effectiveness in securing networks and data, making it a trusted partner for businesses worldwide.

📋 Description

• Lead, mentor and grow a team of researchers and engineers focused on evaluating AI models applied to cybersecurity tasks • Define the strategy, roadmap and success metrics for evaluating AI/LLM and agentic systems across security use cases • Design and standardize evaluation methodologies, benchmark datasets and reproducible testing pipelines rooted in real SOC workflows • Assess models for accuracy, robustness, reliability and operational effectiveness in security-analyst scenarios • Establish quantitative and qualitative metrics for AI models and agentic systems against threat-detection and response challenges • Collaborate with engineering, product and threat-research teams to translate evaluation findings into scalable improvements • Communicate results, trade-offs and recommendations to technical and executive audiences

🎯 Requirements

• Hands-on SOC experience with a strong understanding of day-to-day security operations • At least 3 years in a management or team-leadership position, with a proven track record of mentoring and growing technical teams • Strong knowledge of incident response and threat hunting, including detection, investigation and remediation workflows • Broad knowledge of the cybersecurity landscape, including attack vectors, defense mechanisms, and analyst workflows • Practical experience working with AI, including familiarity with AI/LLM capabilities and knowledge of agentic systems • Ability to define, design and standardize evaluation methodologies and reproducible testing pipelines • Exceptional communication skills, including the ability to present complex technical findings clearly to technical and non-technical audiences • Demonstrated track record of delivering results, supported by shareable projects or measurable outcomes • Proven experience utilizing AI technologies to enhance decision-making, streamline workflows and processes, improve efficiency and drive business outcomes • Relevant security certifications are a bonus, e.g. GCIA, GCIH, GCFA, OSCP, or equivalent • Experience building or working with LLM/agentic evaluation frameworks and benchmarking pipelines • Programming experience in Python and familiarity with data-science tooling • Familiarity with SOC platforms, SIEM/SOAR tools, and detection engineering • Exposure to adversarial testing, red-teaming, or robustness evaluation of AI systems • Understanding of MITRE ATT&CK and how it maps to detection and response workflows

🏖️ Benefits

• Competitive salary • Stock options • Private Healthcare insurance • Life insurance • Training budget • Working with the latest technologies • Flexible time off • Team hangouts • Market leader in compensation and equity awards • Comprehensive physical and mental wellness programs • Competitive vacation and holidays for recharge • Paid parental and adoption leaves • Professional development opportunities for all employees regardless of level or role • Employee Networks, geographic neighborhood groups, and volunteer opportunities to build connections • Vibrant office culture with world class amenities • Great Place to Work Certified™ across the globe