Contra
11 - 50 employees
Founded 2019
🏪 Marketplace
🎯 Recruiter
Marketplace • Recruitment
Contra is a marketplace and network for creative professionals that connects brands and companies with independent creatives (designers, developers, video editors, motion designers, and other creative specialists). It offers commission-free payments, verified studios, hiring tools, community-driven challenges and prize events, and research/AI initiatives through Contra Labs (human-in-the-loop creative data and benchmarks for fine-tuning AI). The platform emphasizes project discovery, vetted freelancers and studios, and community learning (Contra University) to help brands find and hire creative talent and help creatives earn and showcase work.
Research Scientist, Human Data & Evaluation
Job not on LinkedIn
🔥 0 minutes ago
📊 Check your resume score for this job
Improve your chances of getting an interview by checking your resume score before you apply.
Contra
11 - 50 employees
Founded 2019
🏪 Marketplace
🎯 Recruiter
Marketplace • Recruitment
Contra is a marketplace and network for creative professionals that connects brands and companies with independent creatives (designers, developers, video editors, motion designers, and other creative specialists). It offers commission-free payments, verified studios, hiring tools, community-driven challenges and prize events, and research/AI initiatives through Contra Labs (human-in-the-loop creative data and benchmarks for fine-tuning AI). The platform emphasizes project discovery, vetted freelancers and studios, and community learning (Contra University) to help brands find and hire creative talent and help creatives earn and showcase work.
📋 Description
• Translate ambiguous client questions into clear research questions, hypotheses, and study designs • Develop research proposals and methodologies from scratch • Identify new research directions and help define what Contra Labs should measure, study, and build • Design human-data collection strategies, evaluation protocols, rubrics, annotation schemes, and experiments • Develop prompt strategies for eliciting, evaluating, and comparing model behavior • Determine sampling, evaluator selection, calibration, quality-control, and reliability methods • Design methodologies across creative and multimodal domains including design, image, video, and agentic workflows • Analyze qualitative and quantitative evaluation data • Explain why patterns occur and their implications • Synthesize findings into recommendations, research narratives, and hypotheses • Produce client-ready reports, presentations, benchmarks, and research publications • Independently own research engagements from initial scoping through final delivery • Collaborate with project leads, creative experts, research engineers, and data/engineering teams • Maintain research rigor and data quality on fast client timelines • Support multiple concurrent projects at different stages • Participate in client scoping and proposal development • Collaborate with Strategic Project Leads on client delivery
🎯 Requirements
• Based in San Francisco, California • 3–5+ years of relevant research experience • Master’s, PhD, or equivalent industry research depth in HCI, Human-Centered AI, ML, design research, or a related field • Strong qualitative and quantitative research skills • Experience in human evaluation, experimentation, data collection, UX/HCI, annotation, or model behavior research • Ability to independently design and execute rigorous research • Ability to decide what data to collect, why, and how to analyze it • Ability to turn data and observations into insights, research directions, and actionable recommendations • Comfortable analyzing data with Python and/or SQL • Strong written communication and ability to explain methodological tradeoffs and research decisions • Comfortable operating independently through ambiguity and balancing research rigor with startup speed and pragmatism • Familiarity with modern ML and generative AI systems • Understanding of RLHF, post-training, preference data, and model evaluation • Experience studying image, video, multimodal, creative, or agentic systems • Experience designing prompts or systematically studying prompting/model behavior • Experience with large-scale human-data or annotation programs • Experience working directly with AI research or model teams • Strong publication record or evidence of generating original research ideas • Startup / 0-to-1 experience
🏖️ Benefits
• Medical, Dental, Vision Benefits • 401k Matching • Company laptop provided on start date • Equity
Apply NowSimilar Jobs
🕒 4 days ago
Applied Research Scientist designing GPU-native numerical solvers for NVIDIA’s accelerated-computing platforms. Advancing simulation algorithms toward production software and AI-native engineering methods.
🇺🇸 United States – Remote
💵 $192k - $356.5k / year
⏰ Full Time
🟠 Senior
🧬 Research Scientist
🦅 H1B Visa Sponsor
🕒 5 days ago
Computer vision scientist developing Boon’s construction-drawing foundation model. Designing, evaluating, and productionizing multimodal vision systems for construction AI.
🕒 5 days ago
Research Assistant supporting AIR’s education research and technical assistance projects. Preparing data, conducting analyses, coordinating stakeholders, and developing research products for student learning outcomes.
🇺🇸 United States – Remote
💵 $27 - $33 / hour
💰 Grant on 2017-11
⏰ Full Time
🟡 Mid-level
🟠 Senior
🧬 Research Scientist
🦅 H1B Visa Sponsor
🕒 August 10
Research Scientist developing privacy-focused identity matching models for Netflix’s advertising business. Applying machine learning and large-scale data science to improve ad targeting and member experience.
🕒 August 6
R&D Scientist developing and validating NGS analysis algorithms for molecular diagnostics and genomics. Documenting analytical methods and supporting assay development, quality, and regulatory needs.
