Senior Technical Lead, AI Benchmarking and Evaluation Research

🕒 September 8

🇷🇴 Romania – Remote

⏰ Full Time

🟠 Senior

🔴 Lead

🤖 AI Engineer

Where this job comes from

Posted on CrowdStrike's careers page ↗CrowdStrike's careers page, not through a job board

First seen by Remote Rocketship September 8

Still open on CrowdStrike's site. Checked 3 hours ago.

Apply Now
Logo of CrowdStrike

CrowdStrike

5001 - 10000 employees

Founded 2011

🔒 Cybersecurity

☁️ SaaS

🤖 Artificial Intelligence

Cybersecurity • SaaS • Artificial Intelligence

CrowdStrike is a cybersecurity company that provides cloud-based security services to stop breaches. It is recognized as a leader in endpoint protection, identity and cloud security, and managed detection and response. CrowdStrike's platform, Falcon, integrates artificial intelligence to offer real-time visibility, detection, and protection against sophisticated cyber threats. The company is lauded for its effectiveness in securing networks and data, making it a trusted partner for businesses worldwide.

📋 Description

• Design and build benchmark datasets for cybersecurity agent tasks, from task definition through expert-labeled ground truth, scoring rubrics and release • Build and calibrate LLM-based and programmatic judges against expert labels • Build reproducible evaluation harnesses capturing agent actions and traces • Run and publish standing scorecards of CrowdStrike agents and frontier models • Investigate agent failure modes and supporting evidence • Write internal reports and external papers or talks • Lead, mentor and grow a team of researchers and engineers • Set strategy, roadmap and success metrics for AI and agentic system evaluation • Define and enforce evaluation methodology and shared task and trace formats • Coordinate subject matter expert labeling from SOC, threat research and malware analysis teams • Collaborate with engineering, product and threat research on pass/fail criteria and improvements • Communicate results, trade-offs and limitations to technical and executive audiences

🎯 Requirements

• Hands-on cybersecurity expertise in at least one of: malware analysis and reverse engineering, incident response and threat hunting, detection engineering, SOC operations • Demonstrated experience building evaluations, benchmarks or datasets for AI or LLM systems, with shareable artifacts • Strong Python and data tooling skills • Practical experience with LLMs and agentic systems: tool use, multi-step planning, sandboxing, and instrumenting and tracing agent behavior • At least 2 years leading technical people as a team lead, tech lead or manager • Ability to define, design and standardize evaluation methodologies and reproducible testing pipelines • Broad knowledge of the cybersecurity landscape, including attack techniques, defensive controls, and analyst workflows • Exceptional written and spoken communication • Proven experience utilizing AI technologies to enhance decision-making, streamline workflows and processes, improve efficiency and drive business outcomes • Relevant security certifications are a bonus: GREM, GCFA, GCIH, GCIA, OSCP or equivalent

🏖️ Benefits

• Market leader in compensation and equity awards • Comprehensive physical and mental wellness programs • Competitive vacation and holidays for recharge • Paid parental and adoption leaves • Professional development opportunities for all employees regardless of level or role • Employee Networks, geographic neighborhood groups, and volunteer opportunities to build connections • Vibrant office culture with world class amenities • Great Place to Work Certified™ across the globe

Apply Now

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Similar Jobs

🕒 July 31

JUSTROCKET

11 - 50

💼 Consulting

📣 Marketing

☁️ SaaS

Software Engineer developing customer-centric applications with AI focus at JUSTROCKET. Collaborate with a cross-functional team in an agile environment to create innovative solutions.

🗣️🇩🇪 German Required

🕒 July 27

eMAG

5001 - 10000

🛍️ eCommerce

🏪 Marketplace

Agentic AI Engineer building AI-powered workflows that accelerate engineering teams for eMAG. Focused on software engineering and architecture expertise in the rapidly evolving AI landscape.

🕒 April 6

Brillio

1001 - 5000

🏥 Healthcare

🛡️ Insurance

📦 Logistics

Lead AI Engineer designing and implementing AI and computer vision solutions on Microsoft Azure. Working with data science and engineering teams to build production-ready systems.

🇷🇴 Romania – Remote

💰 Private Equity Round on 2019-01

⏰ Full Time

🟡 Mid-level

🟠 Senior

🤖 AI Engineer

Built by Lior Neu-ner. I'd love to hear your feedback — Get in touch via DM or support@remoterocketship.com
Follow Remote Rocketship on LinkedInFollow Remote Rocketship on InstagramFollow Remote Rocketship on FacebookFollow Remote Rocketship on TikTok