Software Engineer L5/L6 – Model Evaluations, Data Curation

🔥 0 minutes ago

🇺🇸 United States – Remote

💵 $600k - $1.1M / year

⏰ Full Time

🟡 Mid-level

🟠 Senior

🧑‍💻 Full-stack Engineer

👻 Ghost score 0%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Netflix

WebsiteLinkedInAll Job Openings

10,000+ employees

Founded 1997

📱 Media

👥 B2C

Media • B2C

Netflix is a global streaming entertainment company and content producer whose stated mission is "to entertain the world. " It operates a consumer-facing subscription platform offering on-demand TV shows, films, and original programming, and also runs a public careers site emphasizing culture, inclusion, and hiring accommodations. The provided text highlights Netflix’s focus on recruiting talent worldwide, its work-life and culture pages, and its public-facing employer materials.

📋 Description

• Design and build shared data curation infrastructure, including reusable components, libraries, and workflows • Build scalable LLM-driven data transformation pipelines using raw sources such as the Netflix catalog and metadata • Use large-scale batch inference with attention to quality and token cost • Develop sampling strategies covering coverage, diversity, difficulty, and balance across content and member segments • Develop filtering and quality-control methods, including LLM-as-judge, evaluation-model-based scoring, deduplication, and validation • Partner with researchers to measure how curation choices affect model performance • Make curated datasets discoverable artifacts with versioning, explicit lineage, and reproducibility • Drive adoption of shared curation practices across MEDC and partner modeling teams

🎯 Requirements

• Strong software engineering in Python, with experience building reusable infrastructure, libraries, or frameworks used by other engineers and researchers • Experience building LLM-driven data generation or transformation pipelines, including synthetic data, structured outputs, or batch inference at scale • Hands-on experience with data quality methods: sampling strategies, filtering, deduplication, and model-based quality scoring such as LLM-as-judge • Understanding of how data choices affect model behavior and ability to design experiments that measure it • Experience with distributed data processing such as Spark, Ray, or similar • Excellent collaboration skills, particularly with researchers, data scientists, and platform teams • Experience with LLM evaluation systems (must-have for L6) • Technical leadership across data and evaluation infrastructure; experience setting technical direction for a multi-engineer effort (must-have for L6) • Experience with dataset versioning, lineage, and artifact management • Experience optimizing cost and throughput for large-scale LLM inference • Experience with human annotation workflows and calibrating LLM judges against human raters • Experience with pipeline orchestration frameworks such as Metaflow, Airflow, or similar • Background in recommendation systems, personalization, search, or working with content catalog and metadata

🏖️ Benefits

• Health Plans • Mental Health support • 401(k) Retirement Plan with employer match • Stock Option Program • Disability Programs • Health Savings and Flexible Spending Accounts • Family-forming benefits • Life and Serious Injury Benefits • Paid leave of absence programs • Full-time hourly employees accrue 35 days annually for paid time off to be used for vacation, holidays, and sick paid time off • Full-time salaried employees are immediately entitled to flexible time off