Senior Applied ML Engineer – Model Router

🔥 7 minutes ago

🇺🇸 United States – Remote

⏰ Full Time

🟠 Senior

🤖 Machine Learning Engineer

👻 Ghost score 12%

What it means: the chance that no one is really hiring for this job.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Cast AI

WebsiteLinkedInAll Job Openings

201 - 500 employees

Founded 2019

☁️ SaaS

🤖 Artificial Intelligence

🤝 B2B

SaaS • Artificial Intelligence • B2B

Cast AI is a SaaS platform that automates Kubernetes application performance, infrastructure, and cost optimization for cloud-native teams. It connects to EKS/AKS/GKE and on-prem clusters with no code changes, uses predictive models and runtime signals to rightsize pods, scale nodes, optimize GPUs and Spot instances, and run approved self-healing runbooks to maintain reliability and reduce cloud spend. Cast AI targets platform, SRE, and FinOps teams and supports AI/LLM workloads, GPU optimization, and real-time workload placement and cost intelligence.

📋 Description

• Own the routing model end to end, including features, objective, training, calibration, and release cadence • Build production-parity evaluations, router-vs-router comparisons, task-family holdouts, and error-barred pass-rate analyses • Convert production outcomes, benchmark rollouts, and override feedback into training signals • Categorize failures into infrastructure and task failures to prevent poisoned labels from reaching the policy • Model task success, token cost, wall-clock speed, concurrency, and cached-input economics • Run shadow deployments, A/B tests, full rollouts, drift monitoring, and retraining • Develop calibrated decisions, abstention under uncertainty, and speed-aware routing • Partner with platform engineers on instrumentation, per-request features and labels, and experiment data paths

🎯 Requirements

• 5+ years in applied ML or ML research • Experience training, evaluating, and shipping models that make real-time decisions, such as ranking, recommendation, bandits, pricing, or LLM routing • Strong Python and SQL at scale, including handling millions of request-level records • Ability to distinguish model improvement from easier evaluation • Understanding of LLM benchmarks and agentic harnesses • Knowledge of TTFT, decode speed, batching, prefix caching, and concurrency • Experience instrumenting systems and using error bars • Python, PyTorch, SQL at scale (ClickHouse), OpenAI-compatible and Anthropic-compatible LLM APIs, benchmark and rollout harnesses, and GitLab CI • Background check may be conducted at the final stage • Must be able to work without visa sponsorship or a work permit provided by Cast AI

🏖️ Benefits

• Competitive salary (depending on the level of experience) • Flexible, remote-first global environment • Equity options • 10% of work time for personal projects or self-improvement • Learning budget for professional and personal development, including access to international conferences and courses • Annual hackathon • Team-building budget and company events • Equipment budget • Extra days off • Quick feedback with a fast-paced workflow

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume