Engineering Lead, Inference Optimization

Job not on LinkedIn

🔥 4 hours ago

🦬 Wyoming – Remote

🦬 Wyoming – Remote

⏰ Full Time

🟠 Senior

🧑‍💻 Full-stack Engineer

👻 Ghost score 15%

What it means: the chance that no one is really hiring for this job.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Venice.ai

WebsiteLinkedInAll Job Openings

51 - 200 employees

Founded 2024

🤖 Artificial Intelligence

☁️ SaaS

🔌 API

🔥 Funding within the last year

💰 $65M Series A on 2026-06

Artificial Intelligence • SaaS • API

Venice. ai is a privacy-focused AI platform that provides access to leading third-party and open-source models for generating text, images, video, audio, code, embeddings, and agents. The company emphasizes user privacy through anonymized access, zero data retention for self-hosted models, Trusted Execution Environments (TEEs), and client-side end-to-end encryption. Venice offers a unified API compatible with OpenAI and agent stacks, developer tools (SDKs, LangChain, OpenAI SDK compatibility), and products for creators and enterprises with tiered pricing plans. Its features include multimodal generation, image and video creation/editing/upscaling, text-to-speech and music generation, and support for uncensored models.

📋 Description

• Own Venice’s technical strategy for inference performance • Recruit and lead the Inference Optimization Team at Venice • Optimize Venice's GPU infrastructure across architectures such as H200s and B300s • Improve latency, throughput, and cost per token for LLM inference workloads • Build reproducible benchmarking harnesses across inference engines such as vLLM and SGLang • Identify the optimal engine, quantization scheme, and parallelism strategy for each workload and GPU SKU • Optimize multivariate inference load-balancing algorithms with the inference routing system • Evaluate emerging inference optimization techniques, attention variants, quantization schemes, and compilation stack improvements • Evaluate emerging inference hardware, including FPGAs, ASICs, and custom silicon • Perform hands-on individual-contributor work while managing a small team and shaping Venice’s technical strategy

🎯 Requirements

• 8+ years in performance optimization or HPC, with deep GPU architecture and parallel programming knowledge • 5+ years experience leading engineering teams • Proficiency in Python, Rust, or Go • Hands-on experience with at least one production LLM inference engine, such as vLLM or SGLang, running at high volume in production • Demonstrated experience with LLM inference optimization techniques: continuous batching, PagedAttention/KV cache management, speculative decoding, quantization, CUDA graphs, and torch.compile • Fluency with quantization tradeoffs, both qualitative and quantitative • Experience with distributed inference strategies, including tensor parallelism, pipeline parallelism, and MoE parallelism, in multi-GPU and multi-node environments • Fluency with GPU profiling using Nsight Systems, Nsight Compute, or PyTorch Profiler • Bonus: C++/CUDA • Bonus: diffusion/image model inference optimization, custom Triton kernels, and contributions to open-source inference frameworks

🏖️ Benefits

• No explicit benefits, perks, or compensation extras stated

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume