Member of Technical Staff, ML Platform – Evals

🕒 September 22

🇺🇸 United States – Remote

💵 $240k - $290k / year

⏰ Full Time

🟡 Mid-level

🟠 Senior

🖥 Software Engineer

Where this job comes from

Posted on Runway's careers page ↗Runway's careers page, not through a job board

First seen by Remote Rocketship September 22

Still open on Runway's site. Checked 6 hours ago.

Apply Now
Logo of Runway

Runway

51 - 200 employees

Founded 2018

🤖 Artificial Intelligence

☁️ SaaS

📱 Media

Artificial Intelligence • SaaS • Media

Runway is a company building foundational "Real-World Intelligence" — large models and systems that can understand, simulate and act in the visual world. It offers three main product platforms built on these models: Runway Creative, an all‑in‑one cloud creative suite for generating and editing video, images and audio for individuals and teams; Runway Dev, an AI media platform and API for developers to run models, compose pipelines and trigger workflows; and Runway Robotics, a toolkit for photorealistic simulation, policy inference and synthetic data generation powered by their General World Model (GWM-1). Runway also publishes research (e. g. , GWM-1, Gen-4. 5 video model) and serves both individual creators and enterprise customers.

📋 Description

• Own the ML evaluation platform end to end, including tooling and systems to generate, annotate, review, and adapt evaluations at frontier scale • Define CLIs, APIs, GUIs, and storage layers for a seamless, fast, sophisticated, and collaborative evaluation process • Work directly with research teams on video, image, audio, agents, and robotics to understand measurement needs and build generalized platform capabilities • Set standards for model evaluation, including reproducibility, metric definitions, reporting formats, and thresholds for trustworthy results • Support broad adoption and integration of the platform across training, production model serving, and other research efforts • Contribute to ML Platform tools and systems that help Runway train and serve frontier models • Generate samples at scale, score them with automated metrics and human annotations, track results across checkpoints and releases, and present findings to researchers quickly • Define operational measures of model quality and support ship/no-ship decisions

🎯 Requirements

• 5+ years of experience building ML infrastructure or data platforms in production environments • Experience with evaluation, experimentation, or benchmarking systems • Strong Python and PyTorch skills • Hands-on experience running large batch GPU workloads on Kubernetes • Experience designing data pipelines and storage for large volumes of media or model outputs • Familiarity with versioning and reproducibility • Familiarity with experimental statistics, including paired comparisons, confidence intervals, multiple-comparison pitfalls, and inter-rater agreement • Comfort building internal tools end to end, from command line to browser • Ability to lead a broad technical area, gather requirements, set direction, make tradeoffs, and drive a roadmap • Familiarity with the full model development lifecycle: data, training, evaluation, and serving • Ability to work as a self-starter embedded with research teams • Strong systems thinking and pragmatic approach to production reliability • Humility and open mindedness • If located in the US, current authorization to work in the US is requested in the application form • Track record of building evaluation suites for generative models is nice to have • Hands-on work with LLM- or VLM-as-judge pipelines is nice to have • Experience with online experimentation platforms and connecting offline metrics to product outcomes is nice to have • Prior work evaluating agents or robotics policies is nice to have

🏖️ Benefits

• Salary range of $240K–$290K • Access to best-in-class models, agents, GPUs, storage and cloud services • Diverse, inclusive workplace with equal opportunity to succeed • Global research and engineering environment

Apply Now

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Similar Jobs

🕒 September 22

Penn Information Systems & Computing

201 - 500

🔌 API

🤝 B2B

🏢 Enterprise

Senior Curriculum Developer creating rigorous science education materials for University of Pennsylvania’s C4R team. Designing, evaluating, and improving training content for scientists.

🕒 September 22

Connor Group

201 - 500

💼 Consulting

🏥 Healthcare

📦 Logistics

Senior MuleSoft developer designing and supporting Workato, MuleSoft, and Dell Boomi integrations. Delivering automation solutions for Connor Group’s finance, accounting, and technology clients.

🕒 September 22

CACI International Inc

10,000+ employees

🎖️ Defense

🏛️ Government

🔒 Cybersecurity

UPK Developer creating Oracle UPK simulations and training materials for CACI's Marine Corps ERP modernization program. Supporting GCSS-MC end-users through accurate, multi-format learning content.

🇺🇸 United States – Remote

💵 $57.5k - $117.9k / year

🔥 Funding within the last year

💰 $500M Post-IPO Debt on 2026-02

⏰ Full Time

🟡 Mid-level

🖥 Software Engineer

🕒 September 22

Accenture Federal Services

10,000+ employees

💼 Consulting

🎖️ Defense

📦 Logistics

Oracle SaaS integration developer designing Oracle Fusion Cloud interfaces for Accenture Federal Services. Supporting US federal agencies with scalable integrations, data flows, and technical documentation.

🕒 September 22

Metrc

51 - 200

💼 Consulting

📦 Logistics

🏥 Healthcare

Senior Programmer supporting California customers using Metrc’s cannabis regulatory tracking software. Managing client services, data delivery, requirements, documentation, onboarding, and customer success.

Built by Lior Neu-ner. I'd love to hear your feedback — Get in touch via DM or support@remoterocketship.com
Follow Remote Rocketship on LinkedInFollow Remote Rocketship on InstagramFollow Remote Rocketship on FacebookFollow Remote Rocketship on TikTok