Research Engineer, QC Automation

Job not on LinkedIn

🔥 6 minutes ago

🏄 California – Remote

🏄 California – Remote

⏰ Full Time

🟡 Mid-level

🟠 Senior

📚 Research Engineer

👻 Ghost score 15%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

hud (YC W25)

WebsiteLinkedInAll Job Openings

1 - 10 employees

Founded 2025

🤖 Artificial Intelligence

☁️ SaaS

🤝 B2B

Artificial Intelligence • SaaS • B2B

hud is an evaluation platform designed for AI agents, facilitating the assessment and iteration of these agents across a variety of environments and tasks. With tools that cater to both researchers and professionals, hud allows users to rapidly evaluate their AI models and integrate existing tools while ensuring swift results through the orchestration of concurrent machine evaluations. Its flexible architecture supports customized tasks and diverse workflows, making it a robust solution for enhancing AI performance and capabilities.

📋 Description

• Build systems to automate quality control for training data created using HUD’s infrastructure • Create QC systems based on true understanding and human judgment, without heavy reliance on LLMs • Define and enforce quality standards for training data • Design experiments and metrics to grade agent outputs • Partner with data vendors to debug quality issues and diagnose agent failure modes • Provide actionable feedback and improve vendor data-generation processes • Translate QC learnings into supplier-dataset auditing systems, including sampling strategies, validation pipelines, and feedback loops • Integrate QC learnings into infrastructure tools and the data vendor portal to reduce anomalies, inconsistencies, and edge cases • Participate in a process involving two technical interviews and a 2–3 day work trial

🎯 Requirements

• Proficiency in Python, Docker, and Linux environments • Strong understanding of what “good data” means and how to measure it • Genuine curiosity of different domains and ability to ask questions to understand them • Experience building scalable data validation pipelines and automated QA/QC systems end-to-end without a fully prescribed roadmap • Experience working on benchmarks and evals, including reasoning about realistic tasks, reliable rubrics, usable environments, and useful RL training trajectories • Early-stage startup experience • Ability to work independently in fast-paced environments • Knowledge of statistics may be beneficial • Strong written and verbal communication skills may be beneficial • Comfort designing metrics, experiments, and QA/QC processes may be beneficial • Experience with existing benchmarks and constructing tasks in new evals may be beneficial • Ability to thrive in unstructured problem spaces • Ability to work hours that overlap 70–80% with either San Francisco or Singapore time zones for remote work • Technical aptitude and learning potential prioritized over years of experience

🏖️ Benefits

• 100% covered top-of-the-line medical, dental, and vision from Blue Shield of CA (US employees) • Lunch and dinner when you’re in the office (in-office employees) • Company-wide holiday break (Christmas Eve to New Year’s Day) on top of PTO and paid holidays • PTO and paid holidays • Equinox membership (US employees) • 401k (US employees) • Commuter benefits (US employees) • Unlimited access to tokens for ChatGPT, Claude Code, Cursor, etc. • Support for relocation and visas for strong full-time candidates to the US or Singapore