Senior Machine Learning Engineer, Inference

Job not on LinkedIn

🔥 1 hour ago

🇷🇸 Serbia – Remote

⏰ Full Time

🟠 Senior

🤖 Machine Learning Engineer

👻 Ghost score 13%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Inworld AI

WebsiteLinkedInAll Job Openings

51 - 200 employees

Founded 2021

💼 Consulting

🤖 Artificial Intelligence

🔌 API

💰 $50M Series A on 2022-08

Consulting • Artificial Intelligence • API

Inworld AI is a developer-focused conversational AI platform that provides a realtime, provider-agnostic runtime and high-quality TTS for building expressive voice agents and virtual characters. It offers multilingual, low-latency streaming voice, instant cloning, emotion and non-verbal controls, and an API-based integration layer with templates and pipelines optimized for scale and cost. The platform is used across games, media, contact centers, and other interactive experiences to boost engagement and reduce infrastructure costs.

📋 Description

• Optimize inference using modern serving frameworks and techniques • Accelerate models through quantization, distillation, caching, continuous batching, paged attention, and speculative decoding • Profile and optimize high-performance code for NVIDIA GPUs • Build and scale distributed inference systems using Kubernetes, Ray, custom load balancing, and multi-GPU/multi-node infrastructure • Handle thousands of concurrent connections reliably • Take models from the research team through containerization, serving optimization, and production deployment • Design benchmarks or prototypes to clarify ambiguous technical problems • Challenge architectures to improve latency and throughput • Ensure engineering work is shipped and stable • Collaborate daily with US-based leadership and engineering teams • Contribute to public technical work and open-source projects where appropriate

🎯 Requirements

• Deep understanding of modern serving frameworks and techniques such as vLLM or TRT-LLM • Hands-on experience with quantization, distillation, caching strategies, continuous batching, paged attention, and speculative decoding • Proficiency in C++, CUDA, Rust, or highly optimized Python • Ability to profile code and optimize performance on NVIDIA GPUs • Experience with Kubernetes, Ray, custom load balancing, multi-GPU/multi-node inference, and handling thousands of concurrent connections • Non-trivial systems programming projects, open-source contributions to major inference engines, or deep-dive technical write-ups • Ability to take a model from the research team through containerization and serving optimization to reliable production deployment • PhD in CS, Physics, or Math, or equivalent practical experience building backend or ML systems • Professional fluency in English, written and spoken • Ability to learn quickly, thrive in ambiguity, work independently, and prioritize impact and stable production delivery

🏖️ Benefits

• Full U.S. visa and relocation support may be available for candidates interested in relocating to the San Francisco Bay Area in the future, subject to business needs and applicable legal and work authorization requirements • Opportunities to share work and make open-source contributions