Senior / Lead Machine Learning Engineer, Inference

🔥 0 minutes ago

🇩🇪 Germany – Remote

⏰ Full Time

🟠 Senior

🤖 Machine Learning Engineer

👻 Ghost score 13%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Inworld AI

WebsiteLinkedInAll Job Openings

51 - 200 employees

Founded 2021

💼 Consulting

🤖 Artificial Intelligence

🔌 API

💰 $50M Series A on 2022-08

Consulting • Artificial Intelligence • API

Inworld AI is a developer-focused conversational AI platform that provides a realtime, provider-agnostic runtime and high-quality TTS for building expressive voice agents and virtual characters. It offers multilingual, low-latency streaming voice, instant cloning, emotion and non-verbal controls, and an API-based integration layer with templates and pipelines optimized for scale and cost. The platform is used across games, media, contact centers, and other interactive experiences to boost engagement and reduce infrastructure costs.

📋 Description

• Build and optimize realtime inference systems for consumer-facing AI applications • Work with first-party speech models, LLM serving, and modular high-volume realtime APIs • Optimize model serving and inference performance using modern serving frameworks and acceleration techniques • Profile and optimize code and NVIDIA GPU workloads for latency and throughput • Scale inference across Kubernetes, Ray, multi-GPU, and multi-node environments • Handle thousands of concurrent connections reliably • Take models from research through containerization, serving optimization, and production deployment • Collaborate daily with US-based leadership and engineering teams • Design benchmarks or prototypes to resolve unclear performance and reliability problems • Contribute to impactful engineering work and, where appropriate, open-source projects and technical write-ups

🎯 Requirements

• Deep understanding of modern serving frameworks and techniques such as vLLM or TRT-LLM • Hands-on experience with quantization, distillation, caching strategies, continuous batching, paged attention, and speculative decoding • Proficiency in C++, CUDA, Rust, or highly optimized Python • Ability to profile code and optimize NVIDIA GPU performance • Experience with Kubernetes, Ray, custom load balancing, multi-GPU/multi-node inference, and handling thousands of concurrent connections • Non-trivial systems programming projects, open-source contributions to major inference engines, or deep-dive technical write-ups • Ability to take a model from the research team through containerization, serving optimization, and reliable production deployment • PhD in CS, Physics, Math, or equivalent practical experience building backend or ML systems • Professional fluency in English, written and spoken, is required • Ability to work legally in Germany, as indicated by the required legal-right-to-work application question

🏖️ Benefits

• Full U.S. visa and relocation support may be available for candidates interested in relocating to the San Francisco Bay Area in the future, subject to business needs and applicable legal and work authorization requirements