
501 - 1000 employees
Founded 2012
💼 Consulting
📣 Marketing
📦 Logistics
Consulting • Marketing • Logistics
G2 is a business software review platform that provides insights to potential buyers through over 2. 5 million real user reviews. The platform covers a wide range of software categories including project management, video conferencing, e-commerce platforms, CRM, AI chatbots, and many more. G2 empowers businesses to discover and compare software options to make informed purchasing decisions. It serves as a resource for marketers, sales professionals, IT and finance departments to optimize their software investments. Additionally, G2 offers solutions for software vendors to enhance their profiles and reach more buyers.
🕒 October 27
🇺🇸 United States – Remote
💵 $190k - $235k / year
⏰ Full Time
🟡 Mid-level
🟠 Senior
🧬 Research Scientist
🦅 H1B Visa Sponsor
👻 Ghost score 0%
Where this job comes from
Posted on G2's careers page ↗, not through a job board
First seen by Remote Rocketship October 27
Still open on G2's site. Checked 4 hours ago.

501 - 1000 employees
Founded 2012
💼 Consulting
📣 Marketing
📦 Logistics
Consulting • Marketing • Logistics
G2 is a business software review platform that provides insights to potential buyers through over 2. 5 million real user reviews. The platform covers a wide range of software categories including project management, video conferencing, e-commerce platforms, CRM, AI chatbots, and many more. G2 empowers businesses to discover and compare software options to make informed purchasing decisions. It serves as a resource for marketers, sales professionals, IT and finance departments to optimize their software investments. Additionally, G2 offers solutions for software vendors to enhance their profiles and reach more buyers.
• Own the evaluation methodology and set standards for evaluating AI agents • Work with engineers and product managers to build and improve evaluations in G2’s current offerings • Scope features, turn methods into production-ready components, and iterate based on customer and business needs • Design realistic environments and task sets reflecting business work • Develop accurate, reproducible and difficult-to-game scoring approaches using automated checks, LLM-based judges and human review • Measure consistency and reliability of evaluations • Audit tasks and graders, quantify noise, and establish when agent differences are real • Present and defend evaluation findings to buyers, customers, partners and the research community • Mentor and guide researchers and engineers on evaluation design and measurement • Review work and document standards for repeatable evaluations • Set and pursue a research agenda across agent evaluation, user simulation, LLM-as-judge reliability, voice and multimodal agents, long-horizon workflows and test-set contamination • Follow new benchmarks, papers and evaluation practices and translate them into product and roadmap decisions • Write methodology documentation, papers and public posts where appropriate • Partner with data and go-to-market teams on presentation and use of results
• Experience working with evaluation design or dataset curation in multi-agentic systems • 5+ years building or evaluating ML systems, including hands-on work with LLMs or LLM-based agents • 2+ years experience managing small to medium teams preferred • A graduate degree (MS or PhD) in machine learning, NLP, computer science, statistics or a related field, or equivalent applied research experience • Track record of original ideas in machine learning, shown through first-author publications or projects led end-to-end • Ability to own a research agenda and carry long-running projects independently • Track record of shipping ML work in a product setting with engineers and product managers • Experience setting technical direction, mentoring researchers or engineers, and reviewing methodology • Familiarity with AI benchmarking and benchmark limitations • Sound statistical judgment, including experimental design, variance, significance and agreement between human and automated raters • Good judgment about practical trade-offs • Clear writing and ability to explain methodology to non-specialists and defend it to skeptical audiences • Experience in agent evaluation, user simulation, or human and AI-based judging is helpful • Experience with conversational, voice, retrieval-based or tool-using AI systems is helpful • Contributions to open-source evaluation tools or public leaderboards are helpful • Familiarity with business software workflows is helpful • Publications at ACL, EMNLP, NeurIPS, ICML, ICLR, COLM or similar venues are helpful
• Equity • Flexible work • Ample parental leave • Unlimited PTO
Apply NowImprove your chances of getting an interview by checking your resume score before you apply.
🕒 October 26
Principal Scientist leading AI-driven discovery of rare-earth-lean permanent magnets. Guiding simulation, experimental validation, IP, and U.S. manufacturing partnerships.
🕒 October 23
Translational medicine scientist advancing biomarker strategies for GondolaBio’s genetic-disease therapeutics. Leading assay validation and clinical implementation across early neuroscience programs.
🇺🇸 United States – Remote
💵 $190k - $210k / year
💰 $300M Venture Round - GondolaBio on 2024-08
⏰ Full Time
🟠 Senior
🧬 Research Scientist
🕒 October 20
Applied Research Scientist building Zillow’s AI World Model for housing-market demand and planning. Developing uncertainty-aware models that power next-generation home-shopping experiences.
🇺🇸 United States – Remote
💵 $185.8k - $312.4k / year
💰 $4.1M Post-IPO Equity on 2012-12
⏰ Full Time
🟠 Senior
🧬 Research Scientist
🦅 H1B Visa Sponsor
🕒 October 20
Research Scientist advancing AI-driven voice, fraud, identity, and deepfake security at Pindrop. Developing deep learning models and translating research into production-ready enterprise solutions.
🇺🇸 United States – Remote
⏰ Full Time
🟢 Junior
🟡 Mid-level
🟠 Senior
🧬 Research Scientist
🦅 H1B Visa Sponsor
🕒 October 20
Senior Researcher exploring models, datasets, and experiments for Troveo’s licensed AI training-data platform. Identifying multimodal data opportunities and guiding product and engineering decisions.