Senior Technical Lead, AI Benchmarking and Evaluation Research

🕒 September 8

🇷🇴 Romania – Remote

⏰ Full Time

🟠 Senior

🤖 AI Engineer

👻 Ghost score 24%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

CrowdStrike

WebsiteLinkedInAll Job Openings

5001 - 10000 employees

Founded 2011

🔒 Cybersecurity

☁️ SaaS

🤖 Artificial Intelligence

Cybersecurity • SaaS • Artificial Intelligence

CrowdStrike is a cybersecurity company that provides cloud-based security services to stop breaches. It is recognized as a leader in endpoint protection, identity and cloud security, and managed detection and response. CrowdStrike's platform, Falcon, integrates artificial intelligence to offer real-time visibility, detection, and protection against sophisticated cyber threats. The company is lauded for its effectiveness in securing networks and data, making it a trusted partner for businesses worldwide.

📋 Description

• Design and build benchmark datasets for cybersecurity agent tasks, from task definition through expert-labeled ground truth, scoring rubrics and release • Build and calibrate LLM-based and programmatic judges against expert labels • Build reproducible evaluation harnesses capturing agent actions and traces • Run and publish standing scorecards of CrowdStrike agents and frontier models • Investigate agent failure modes and supporting evidence • Write internal reports and external papers or talks • Lead, mentor and grow a team of researchers and engineers • Set strategy, roadmap and success metrics for AI and agentic system evaluation • Define and enforce evaluation methodology and shared task and trace formats • Coordinate subject matter expert labeling from SOC, threat research and malware analysis teams • Collaborate with engineering, product and threat research on pass/fail criteria and improvements • Communicate results, trade-offs and limitations to technical and executive audiences

🎯 Requirements

• Hands-on cybersecurity expertise in at least one of: malware analysis and reverse engineering, incident response and threat hunting, detection engineering, SOC operations • Demonstrated experience building evaluations, benchmarks or datasets for AI or LLM systems, with shareable artifacts • Strong Python and data tooling skills • Practical experience with LLMs and agentic systems: tool use, multi-step planning, sandboxing, and instrumenting and tracing agent behavior • At least 2 years leading technical people as a team lead, tech lead or manager • Ability to define, design and standardize evaluation methodologies and reproducible testing pipelines • Broad knowledge of the cybersecurity landscape, including attack techniques, defensive controls, and analyst workflows • Exceptional written and spoken communication • Proven experience utilizing AI technologies to enhance decision-making, streamline workflows and processes, improve efficiency and drive business outcomes • Relevant security certifications are a bonus: GREM, GCFA, GCIH, GCIA, OSCP or equivalent

🏖️ Benefits

• Market leader in compensation and equity awards • Comprehensive physical and mental wellness programs • Competitive vacation and holidays for recharge • Paid parental and adoption leaves • Professional development opportunities for all employees regardless of level or role • Employee Networks, geographic neighborhood groups, and volunteer opportunities to build connections • Vibrant office culture with world class amenities • Great Place to Work Certified™ across the globe