Research Scientist, Human Data & Evaluation

Job not on LinkedIn

🕒 August 17

🏄 California – Remote

🏄 California – Remote

💵 $150k - $200k / year

⏰ Full Time

🟡 Mid-level

🟠 Senior

🧬 Research Scientist

👻 Ghost score 15%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Contra

WebsiteLinkedInAll Job Openings

11 - 50 employees

Founded 2019

🏪 Marketplace

🎯 Recruiter

Marketplace • Recruitment

Contra is a marketplace and network for creative professionals that connects brands and companies with independent creatives (designers, developers, video editors, motion designers, and other creative specialists). It offers commission-free payments, verified studios, hiring tools, community-driven challenges and prize events, and research/AI initiatives through Contra Labs (human-in-the-loop creative data and benchmarks for fine-tuning AI). The platform emphasizes project discovery, vetted freelancers and studios, and community learning (Contra University) to help brands find and hire creative talent and help creatives earn and showcase work.

📋 Description

• Translate ambiguous client questions into clear research questions, hypotheses, and study designs • Develop research proposals and methodologies from scratch • Identify new research directions and help define what Contra Labs should measure, study, and build • Design human-data collection strategies, evaluation protocols, rubrics, annotation schemes, and experiments • Develop prompt strategies for eliciting, evaluating, and comparing model behavior • Determine sampling, evaluator selection, calibration, quality-control, and reliability methods • Design methodologies across creative and multimodal domains including design, image, video, and agentic workflows • Analyze qualitative and quantitative evaluation data • Explain why patterns occur and their implications • Synthesize findings into recommendations, research narratives, and hypotheses • Produce client-ready reports, presentations, benchmarks, and research publications • Independently own research engagements from initial scoping through final delivery • Collaborate with project leads, creative experts, research engineers, and data/engineering teams • Maintain research rigor and data quality on fast client timelines • Support multiple concurrent projects at different stages • Participate in client scoping and proposal development • Collaborate with Strategic Project Leads on client delivery

🎯 Requirements

• Based in San Francisco, California • 3–5+ years of relevant research experience • Master’s, PhD, or equivalent industry research depth in HCI, Human-Centered AI, ML, design research, or a related field • Strong qualitative and quantitative research skills • Experience in human evaluation, experimentation, data collection, UX/HCI, annotation, or model behavior research • Ability to independently design and execute rigorous research • Ability to decide what data to collect, why, and how to analyze it • Ability to turn data and observations into insights, research directions, and actionable recommendations • Comfortable analyzing data with Python and/or SQL • Strong written communication and ability to explain methodological tradeoffs and research decisions • Comfortable operating independently through ambiguity and balancing research rigor with startup speed and pragmatism • Familiarity with modern ML and generative AI systems • Understanding of RLHF, post-training, preference data, and model evaluation • Experience studying image, video, multimodal, creative, or agentic systems • Experience designing prompts or systematically studying prompting/model behavior • Experience with large-scale human-data or annotation programs • Experience working directly with AI research or model teams • Strong publication record or evidence of generating original research ideas • Startup / 0-to-1 experience

🏖️ Benefits

• Medical, Dental, Vision Benefits • 401k Matching • Company laptop provided on start date • Equity