Senior Inference Engineer

🕒 August 11

🌐 United States, Canada – Remote

🇺🇸 United States – Remote 🇨🇦 Canada – Remote

💵 $190k - $220k / year

⏰ Full Time

🟠 Senior

👷🏻‍♀️ Engineer

👻 Ghost score 39%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

vCluster

WebsiteLinkedInAll Job Openings

51 - 200 employees

☁️ SaaS

🏢 Enterprise

💰 $24M Series A on 2024-05

SaaS • Enterprise

vCluster is a Kubernetes virtualization product (from loft-sh) that creates virtual Kubernetes clusters on top of existing Kubernetes infrastructure. It enables dedicated, multi-tenant or private-node clusters for use cases like internal platform standardization, hybrid Kubernetes deployments, sovereign or dedicated customer environments, AI cloud providers and distributed inference. vCluster is positioned as a tooling/product solution for enterprises operating Kubernetes at scale, offering managed features, releases and integrations for internal K8s platforms and cloud-native workflows.

📋 Description

• Deploy LLMs to production across one or more GPU-equipped machines • Own the full customer query-to-served-response pipeline • Stand up and operate inference serving infrastructure using vLLM, SGLang, or TensorRT-LLM • Optimize inference at scale through quantization, batching, caching, and routing • Build production infrastructure in Python or Golang • Build the first inference-platform iteration with the CTO • Lead the inference platform roadmap and partner with Product on future development

🎯 Requirements

• Production LLM serving experience using vLLM, SGLang, or TensorRT-LLM • Hands-on experience with quantization, batching, caching, and routing • Strong engineering skills in Python or Golang • Real production code experience • Strong communication skills • Familiarity with Docker and Kubernetes • Hands-on generative AI experience with PyTorch and Transformers • Understanding of CUDA, NCCL, GPU drivers, and related libraries • Knowledge of model architectures and fine-tuning approaches • Experience with NVIDIA Dynamo

🏖️ Benefits

• Equity • Bonus • Health insurance • Dental insurance • Vision insurance • Life insurance • Coverage for eligible dependents • Flexible working schedule • Workplace flexibility • Remote-first culture