
51 - 200 employees
π€ Artificial Intelligence
π€ B2B
βοΈ SaaS
π° Venture Round on 2022-04
Artificial Intelligence β’ B2B β’ SaaS
Robots & Pencils is a digital innovation firm that specializes in developing digital strategies and products utilizing mobile, web, and frontier technologies to drive transformation for businesses. They focus on areas such as mobile development, product design, UX research, artificial intelligence, machine learning, and organizational change management. With a commitment to blending creativity with technology, they aim to unlock data insights and enhance product innovation, ultimately delivering excellent customer experiences and helping brands gain a competitive edge.
π November 12
πΊπΈ United States β Remote
π΅ $200k - $300k / year
β° Full Time
π Senior
π΄ Lead
𧬠Research Scientist
π» Ghost score 5%
Where this job comes from
Posted on Robots & Pencils's careers page β, not through a job board
First seen by Remote Rocketship November 12
Still open on Robots & Pencils's site. Checked 2 hours ago.

51 - 200 employees
π€ Artificial Intelligence
π€ B2B
βοΈ SaaS
π° Venture Round on 2022-04
Artificial Intelligence β’ B2B β’ SaaS
Robots & Pencils is a digital innovation firm that specializes in developing digital strategies and products utilizing mobile, web, and frontier technologies to drive transformation for businesses. They focus on areas such as mobile development, product design, UX research, artificial intelligence, machine learning, and organizational change management. With a commitment to blending creativity with technology, they aim to unlock data insights and enhance product innovation, ultimately delivering excellent customer experiences and helping brands gain a competitive edge.
β’ Design the end-to-end evaluation workflow for a planner-executor agent β’ Define scenario schemas, grading specifications and validators with the clientβs team β’ Build an LLM-based user simulator for realistic multi-turn conversations β’ Build isolated, parallel experiment execution with configuration files and harness adapters β’ Implement optimization levers such as cascade-on-failure, cache-aware model switching and prompt caching β’ Tag model calls and send traces to attribute tokens, cache usage, latency and cost β’ Grade runs using expected end states and optional LLM judges β’ Compute verified success, cost per success, p95 latency and pipeline-stage failures β’ Compare experiments using confidence intervals, paired methods, win/loss/tie analysis and non-inferiority tests β’ Project dollar savings at production volume and plot cost-versus-quality frontiers β’ Build an automated evidence-linked insight engine β’ Build a locally hosted, API-first dashboard with one-command deployment β’ Run staged experiments from production baseline through model and lever screening β’ Analyze generalization by scenario category β’ Deliver recommendations, runbooks and integration guides β’ Partner with client engineers and leadership on scenarios, margins and success criteria β’ Write technical reports and leadership summaries β’ Set experimental standards and mentor engineers on rigorous agent evaluation
β’ Experience with LLM and agent evaluation methods, including programmatic state-based verifiers, LLM-as-judge or agent-as-judge, trajectory-level scoring and user simulation β’ Experience building or evaluating planner-executor or other multi-step, tool-using agent architectures in production β’ Strong background in experimental design and statistical inference, including paired designs, non-inferiority testing and power analysis β’ Experience with multi-objective optimization or cost-aware ML, including Pareto frontier analysis and projected dollar savings β’ Familiarity with LLM routing, model cascades, model selection and LLM inference cost levers β’ Experience with observability and tracing for LLM systems, attributing tokens, cache usage, latency and cost per request β’ Ability to build production-quality research tooling using Python services, containerized and sandboxed execution, and data pipelines β’ Prior customer-facing or consulting experience scoping and delivering research engagements β’ Ability to explain technical findings clearly to non-technical audiences and translate them into business impact
Apply NowImprove your chances of getting an interview by checking your resume score before you apply.
π November 11
Senior Applied Scientist developing production machine learning models for Upstartβs AI lending marketplace. Improving credit prediction accuracy through applied research, experimentation, and statistical analysis.
πΊπΈ United States β Remote
π΅ $167.7k - $231.8k / year
β° Full Time
π‘ Mid-level
π Senior
𧬠Research Scientist
π¦ H1B Visa Sponsor
π November 11
Staff Applied Scientist leading machine learning strategy and modeling for Upstartβs AI lending marketplace. Improving predictive performance, reliability, and scalable credit decisions.
πΊπΈ United States β Remote
π΅ $199.5k - $276k / year
β° Full Time
π‘ Mid-level
π Senior
𧬠Research Scientist
π¦ H1B Visa Sponsor
π November 11
Research Scientist developing personalized recommendation systems for Hook, a social music platform. Applying machine learning research to improve discovery, fan expression, and creator growth.
πΊπΈ United States β Remote
π° $3M Seed Round - Hook on 2025-03
β° Full Time
π Senior
𧬠Research Scientist
π¦ H1B Visa Sponsor
π November 10
Senior Applied Scientist developing and evaluating LLM and machine-learning systems for Zillowβs real estate platform. Shipping reliable AI features that improve home-search customer outcomes.
πΊπΈ United States β Remote
π΅ $164.4k - $262.6k / year
π° $4.1M Post-IPO Equity on 2012-12
β° Full Time
π Senior
𧬠Research Scientist
π¦ H1B Visa Sponsor
π November 10
Senior Medicaid researcher leading policy studies, administrative-data analytics, and client projects for AIR, a nonprofit behavioral and social science research organization.
πΊπΈ United States β Remote
π΅ $120k - $150k / year
π° Grant on 2017-11
β° Full Time
π‘ Mid-level
π Senior
𧬠Research Scientist
π¦ H1B Visa Sponsor