Machine Learning Engineer, Evaluation

Job not on LinkedIn

🔥 16 hours ago

🇺🇸 United States – Remote

💵 $150k - $175k / year

⏰ Full Time

🟢 Junior

🟡 Mid-level

🤖 Machine Learning Engineer

🚫👨‍🎓 No degree required

Where this job comes from

Posted on mpathic's careers page ↗mpathic's careers page, not through a job board

First seen by Remote Rocketship 16 hours ago

Still open on mpathic's site. Checked 12 minutes ago.

Apply Now
Logo of mpathic

mpathic

11 - 50 employees

Founded 2021

🏥 Healthcare

💼 Consulting

🍽️ Food & Beverage

Healthcare • Consulting • Food & Beverage

mpathic is an AI-driven safety and analytics company that offers SaaS products and APIs to record, transcribe, analyze, and improve human–AI and human–human interactions in clinical, life‑science, and enterprise settings. Their offerings include speech-to-text transcription, PII redaction, behavior and checklist analytics, human-curated benchmarking datasets, red‑teaming expertise, and an Observing Agent API to add domain-informed guardrails for healthcare and medical agents. mpathic targets life sciences, clinical research organizations, healthcare providers, insurers, and HR/enterprise teams focused on empathy, safety, bias reduction, and compliance.

📋 Description

• Build, deploy, and maintain ML models and services in collaboration with engineering and product teams • Design evaluation approaches that reflect product needs and real-world operating conditions • Create and maintain evaluation datasets, benchmarks, and automated regression tests that measure safety, empathy, and risk • Select appropriate metrics, analyze performance across meaningful data segments, and investigate failure modes • Design experiments and use statistical methods to judge whether an improvement is real • Connect offline evaluation results with what customers see in production • Develop tools that make evaluation repeatable and useful throughout development and release workflows • Communicate findings, limitations, and recommendations clearly to clinicians, product leads, and customers

🎯 Requirements

• Practical experience developing and shipping ML systems, typically 2–5 years of relevant work or equivalent • Strong Python skills and familiarity with common ML frameworks • Solid ML fundamentals, including training, validation, generalization, and common sources of model error • Experience with evaluation metrics, benchmark design, and systematic error analysis • An understanding of statistical inference, sampling, confidence intervals, and experimental design • Experience identifying data leakage, evaluation bias, and gaps in test coverage • Good software engineering practices, including testing, version control, and code review • Skill at turning ambiguous product goals into measurable quality criteria • A/B testing and production performance monitoring • Human evaluation, annotation guidelines, and inter-rater agreement • Speech or conversational data: transcripts, diarization and multi-speaker setups • Evaluating ranking, retrieval, recommendation, or generative AI systems • Robustness testing, calibration, fairness analysis, or distribution shift detection • Building evaluation tooling and integrating quality checks into CI/CD workflows • Clear, direct communication, both written and verbal • Startups and cross-functional teams

Apply Now

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Similar Jobs

🕒 Yesterday

Humana

10,000+ employees

🏥 Healthcare

🛡️ Insurance

⚕️ Healthcare Insurance

Senior MLOps Engineer building production ML infrastructure for Humana’s healthcare recommendation platform. Deploying, monitoring, and improving reinforcement-learning decision workflows for millions of members.

🕒 3 days ago

Path Robotics

201 - 500

🏗️ Construction

💼 Consulting

📦 Logistics

Machine Learning Engineer building production AI for Path Robotics’ industrial robots. Training and deploying models for robot planning, scanning, and behavior on live robotic cells.

🕒 4 days ago

Augeo

501 - 1000

🤝 B2B

☁️ SaaS

📣 Marketing

Senior Machine Learning Engineer owning production ML, ranking, personalization, and experimentation. Improving reward redemption and engagement for Fortune 500 loyalty clients through Augeo’s THEO AI platform.

🕒 4 days ago

StackAdapt

1001 - 5000

📣 Marketing

☁️ SaaS

📱 Media

Staff ML Scientist building online identity graphs and production models for StackAdapt’s programmatic advertising platform. Improving ad targeting accuracy, reach, and campaign ROI.

🇺🇸 United States – Remote

💵 $182.4k - $250.8k / year

💰 $235M Private Equity Round - StackAdapt on 2025-02

⏰ Full Time

🟡 Mid-level

🟠 Senior

🤖 Machine Learning Engineer

🕒 4 days ago

BetMGM

501 - 1000

💼 Consulting

📣 Marketing

✈️ Travel

Sr. MLOps Engineer building production ML infrastructure on AWS and Snowflake. Enabling reliable model deployment for BetMGM sports betting and online gaming platforms.

🇺🇸 United States – Remote

💵 $117.1k - $156.2k / year

💰 $25.1k Seed Round on 2022-10

⏰ Full Time

🟡 Mid-level

🟠 Senior

🤖 Machine Learning Engineer

Developed by Lior Neu-ner. I'd love to hear your feedback — Get in touch via DM or support@remoterocketship.com
Follow Remote Rocketship on LinkedInFollow Remote Rocketship on InstagramFollow Remote Rocketship on FacebookFollow Remote Rocketship on TikTok