Program Manager, Human Evaluations – Recommendations

🕒 October 20

🇺🇸 United States – Remote

💵 $230k - $360k / year

⏰ Full Time

🟡 Mid-level

🟠 Senior

📋 Program Manager

Where this job comes from

Posted on Netflix's careers page ↗Netflix's careers page, not through a job board

First seen by Remote Rocketship October 20

Still open on Netflix's site. Checked 7 hours ago.

Apply Now
Logo of Netflix

Netflix

10,000+ employees

Founded 1997

📱 Media

👥 B2C

Media • B2C

Netflix is a global streaming entertainment company and content producer whose stated mission is "to entertain the world. " It operates a consumer-facing subscription platform offering on-demand TV shows, films, and original programming, and also runs a public careers site emphasizing culture, inclusion, and hiring accommodations. The provided text highlights Netflix’s focus on recruiting talent worldwide, its work-life and culture pages, and its public-facing employer materials.

📋 Description

• Lead end-to-end execution of human evaluation and data operations initiatives from intake and scoping to delivery • Partner with Product Management to translate recommendation strategy and roadmap priorities into evaluation designs, success metrics, and timelines • Manage human annotation vendors, including capacity planning, SLAs, quality performance, and escalations • Drive operational efficiency across evaluation programs, including cost per task, throughput, turnaround time, and speed to IRR • Develop clear, scalable guidelines and scoring rubrics for consistent rater judgment • Oversee rater onboarding, calibration, and QA workflows • Pilot and refine evaluation tasks to improve clarity, inter-rater reliability, and feedback quality • Anticipate and resolve bottlenecks and blockers • Coordinate product, ML, data science, and vendor partners for aligned execution of evaluations at scale • Act as a thought partner to Product Management and help build shared language around recommendation quality

🎯 Requirements

• 4+ years of experience working in human evaluations, data collection, labeling, or annotation operations in ML environments • Track record of implementing process improvements or quality control systems for data collection needs • Prior experience managing human annotation vendors, raters, or data labeling teams, including vendor performance and cost management • Experience improving operational efficiency of annotation programs, including throughput, turnaround, and cost • Strong understanding of evaluation design, including guidelines, rubrics, and scoring protocols • Working fluency in film and TV content, including genres, formats, and talent • Proven ability in end-to-end management of complex, cross-functional programs • Strong Program Management skills and clear accountability for successful delivery • Experience with human labeling platforms • Excellent written and verbal communication skills • Ability to synthesize feedback into clear recommendations and process improvements • Familiarity with rater subjectivity and bias in taste-based judgments and evaluation design • Strong organizational skills and executional focus • Experience with personalization, recommender systems, search, or ranking evaluation is nice to have • Familiarity with offline human evaluation, online experimentation, and engagement metrics is nice to have • Experience evaluating at the list or page level is nice to have • Experience partnering with Product Managers and Data Scientists on metric definition is nice to have

🏖️ Benefits

• Health Plans • Mental Health support • 401(k) Retirement Plan with employer match • Stock Option Program • Disability Programs • Health Savings and Flexible Spending Accounts • Family-forming benefits • Life and Serious Injury Benefits • Paid leave of absence programs • Flexible time off for full-time salaried employees

Apply Now

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Similar Jobs

🕒 October 20

Valeris

1001 - 5000

💊 Pharmaceuticals

🏥 Healthcare

🤝 B2B

Program Manager coordinating clinical trial support, budgets, vendors, and client relationships for Valeris’ life sciences commercialization solutions. Remote US role based in North Carolina.

🕒 October 20

Affirm

1001 - 5000

💳 Fintech

👥 B2C

🛍️ eCommerce

Senior Program Manager leading global BPO vendor performance, governance, and transformation. Improving customer experience, compliance, and operational scale for Affirm’s point-of-sale financial products.

🕒 October 20

Nametag

11 - 50

🏥 Healthcare

🛡️ Insurance

📦 Logistics

Customer Experience Program Manager scaling support operations for Nametag’s secure digital identity platform. Managing ticket routing, SLAs, L1 support, engineering escalations, and AI-enabled CX workflows.

🇺🇸 United States – Remote

💵 $100k - $125k / year

💰 Series unknown on 2021-02

⏰ Full Time

🟡 Mid-level

📋 Program Manager

🕒 October 20

Aptive Resources

501 - 1000

🎖️ Defense

📦 Logistics

📣 Marketing

Senior Program Analyst supporting the Department of Veterans Affairs’ QUERI research portfolio. Measuring project impacts, managing program infrastructure, and improving processes with Claude, REDCap, and Excel.

🕒 October 20

Bellese Technologies

51 - 200

🏥 Healthcare

💼 Consulting

⚕️ Healthcare Insurance

Program Manager leading technology service delivery, client outcomes, and growth initiatives for Bellese’s civic healthcare solutions. Advising clients and coordinating complex technology and data projects.

Developed by Lior Neu-ner. I'd love to hear your feedback — Get in touch via DM or support@remoterocketship.com
Follow Remote Rocketship on LinkedInFollow Remote Rocketship on InstagramFollow Remote Rocketship on FacebookFollow Remote Rocketship on TikTok