Head of Evaluation, Model Quality

Job not on LinkedIn

🕒 3 days ago

🇺🇸 United States – Remote

⏰ Full Time

🔴 Lead

Where this job comes from

Posted on Subconscious AI's careers page ↗Subconscious AI's careers page, not through a job board

First seen by Remote Rocketship 3 days ago

Still open on Subconscious AI's site. Checked 3 minutes ago.

Apply Now
Logo of Subconscious AI

Subconscious AI

11 - 50 employees

💼 Consulting

🏥 Healthcare

📣 Marketing

💰 Non Equity Assistance on 2023-08

Consulting • Healthcare • Marketing

Subconscious AI is a company that specializes in conducting causal market research using advanced artificial intelligence techniques. They utilize AI agents and synthetic respondents to simulate user behavior and improve market research accuracy. With a digital twin model of consumers, they conduct instant user journey simulations and causal experiments to understand consumer behavior and predict market impacts. They emphasize cost-efficiency and accuracy by replacing human involvement in data collection with AI-driven processes, which allows them to generate insights rapidly. Their services extend to product feature evaluation, audience segmentation, policy simulation, healthcare optimization, and content creation.

📋 Description

• Own evaluation strategy, benchmark governance, regression criteria, and model-quality hiring plan • Define evidence required for release decisions and escalate unsupported claims • Partner with causal research on validity and engineering on reliable evaluation execution • Set the functional roadmap and propose staffing and spending priorities with the founding team • Inventory model claims and customer use cases and map them to testable questions and appropriate baselines • Build reproducible evaluation sets with provenance, held-out testing, leakage controls, and uncertainty estimates • Establish failure analysis across populations, decision types, and distribution shifts • Create release reviews documenting tradeoffs among quality, latency, cost, and known limitations • Track reproducibility, regression detection, consequential-use-case coverage, benchmark stability, cohort error, and verified-improvement timelines • Collaborate directly with founders across scientific and commercial boundaries • Perform early evaluation work hands-on

🎯 Requirements

• Experience leading evaluation, measurement, or model-quality programs • Evidence of decisions changed because of evaluation evidence • Strong statistics and practical machine-learning evaluation skills • Hands-on ability to build data and analysis pipelines • Ability to investigate confounding or leakage • Ability to assess metric usefulness and explain findings • Evidence of judgment, execution, and leadership • Must be able to work in the United States • Current role does not include visa sponsorship

🏖️ Benefits

• Only equity-based compensation is currently available; no cash salary is available today • Equity grant terms to be agreed before joining • Any future salary, incentive plan, benefits, hiring budget, or paid start date depends on financing and written agreement • Remote work • Full-time planned role • Direct collaboration with founders • Working arrangement and travel expectations to be discussed

Apply Now

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Similar Jobs

🕒 3 days ago

Teladoc Health

5001 - 10000

🏥 Healthcare

👥 B2C

☁️ SaaS

Head of Cardiometabolic Solutions leading Teladoc Health’s diabetes, obesity, hypertension, and GLP-1 virtual-care portfolio. Driving product strategy, commercialization, clinical value, and B2B growth.

🇺🇸 United States – Remote

💵 $210k - $230k / year

💰 $80M Post-IPO Debt - Teladoc Health on 2016-07

⏰ Full Time

🔴 Lead

🕒 3 days ago

Flow Pouches

2 - 10

🧘 Wellness

🛍️ eCommerce

👥 B2C

Head of Organic Strategy building Flow Pouch’s organic audience and content engine. Driving views into subscriptions for a functional mushroom pouch brand.

🕒 3 days ago

ShopMy

51 - 200

📣 Marketing

🛍️ eCommerce

☁️ SaaS

Trust and Standards leader scaling safeguards for ShopMy’s creator-commerce platform. Owning policy, escalations, risk vendors, and executive trust reporting.

🇺🇸 United States – Remote

💵 $200k - $250k / year

⏰ Full Time

🟠 Senior

🔴 Lead

🕒 3 days ago

Prenetics

51 - 200

🏥 Healthcare

🍽️ Food & Beverage

📣 Marketing

Head of North America Fulfillment for IM8, a health and longevity nutrition brand. Leading fulfillment quality, compliance and delivery across DTC, Amazon, B2B and retail.

🕒 6 days ago

Bask Health

11 - 50

🏥 Healthcare

💼 Consulting

📦 Logistics

Head of Customer Outcomes leading customer success for Bask Health's direct-to-consumer telehealth software platform. Driving adoption, retention, renewals, and account expansion.

🇺🇸 United States – Remote

💰 Pre Seed Round on 2022-05

⏰ Full Time

🔴 Lead

Developed by Lior Neu-ner. I'd love to hear your feedback — Get in touch via DM or support@remoterocketship.com
Follow Remote Rocketship on LinkedInFollow Remote Rocketship on InstagramFollow Remote Rocketship on FacebookFollow Remote Rocketship on TikTok