Distinguished Engineer, Scaled Out Inferencing

🕒 August 11

🏄 California – Remote

🏄 California – Remote

💵 $320k - $488.8k / year

⏰ Full Time

🟠 Senior

🔴 Lead

👷🏻‍♀️ Engineer

🦅 H1B Visa Sponsor

This company has sponsored H1B visas in the past.

👻 Ghost score 13%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

NVIDIA

WebsiteLinkedInAll Job Openings

10,000+ employees

Founded 1993

🏥 Healthcare

🏭 Manufacturing

🤖 Artificial Intelligence

Healthcare • Manufacturing • Artificial Intelligence

NVIDIA is a leading technology company specializing in accelerated computing and artificial intelligence. NVIDIA pioneers advancements in graphical processing units (GPUs), cloud computing, data centers, and virtual reality, with a focus on gaming, automotive, healthcare, and robotics industries. The company's innovations, such as NVIDIA Omniverse, transform traditional digital processes by enabling high-fidelity simulations and rendering tasks. Their applications span various industries, from autonomous vehicles using NVIDIA DRIVE to healthcare solutions with NVIDIA Clara, and AI-driven analytics and workflows.

📋 Description

• Architect distributed pipelines and drive technical implementation of high-throughput, low-latency distributed inference systems for massive-scale AI workloads • Lead hardware-software co-optimization and performance tuning at the kernel and driver level • Optimize GPU resource management and hardware acceleration for production-grade model serving • Guide and influence open source projects including Dynamo, TensorRT-LLM, vLLM, SGLang, Linux, Kubernetes, and Ray • Orchestrate full model lifecycles, including automated deployment, versioning, and intelligent scaling across cloud and datacenter environments • Collaborate with customers, infrastructure providers, and partners to establish performance and availability standards • Lead technical planning and continuous evolution across ideation, architecture, design, development, deployment, operations, and lifecycle management • Develop organizational alignment across technical and senior corporate leadership • Synthesize cross-functional needs into architecture and design while guiding execution across diverse teams

🎯 Requirements

• 16+ overall years in technical roles • Recent long-term focus on AI infrastructure • Recent direct experience in large-scale inference orchestration • Proven track record building secure, highly available, and durable production distributed systems • 7–10+ years of leadership experience • BS/MS or higher or equivalent experience in systems/software engineering or related engineering fields • Proficiency in GPU architecture, hardware acceleration, and low-level performance tuning, including CUDA and kernels • Experience with cloud-native architectures for multi-tenant model serving • Proven success delivering technically complex solutions with transparency into resource utilization, performance, and operational insights • Ability to develop consensus and organizational alignment across technical and senior corporate leadership • Ability to synthesize cross-functional needs into architecture and design and guide execution across diverse teams • Strong collaboration and influence skills • Real-world experience building systems supporting AI/ML workloads • Direct experience designing, developing, delivering, and operating secure, highly available, scaled-out systems in enterprise and cloud environments • Demonstrated history creating scalable processes and extensible systems for cross-functional collaboration and operations at scale • Familiarity with open source ecosystems and projects including Dynamo, TensorRT-LLM, vLLM, SGLang, and Ray • Ability to collaborate and influence in open source project governance

🏖️ Benefits

• Competitive salaries • Generous benefits package • Equity • Benefits