Senior AI Inference Engineer

🔥 1 hour ago

🇺🇸 United States – Remote

⏰ Full Time

🟠 Senior

🤖 Artificial Intelligence

Where this job comes from

Posted on StackYak's careers page ↗StackYak's careers page, not through a job board

First seen by Remote Rocketship 1 hour ago

Apply Now
Logo of StackYak

StackYak

2 - 10 employees

🤖 Artificial Intelligence

☁️ SaaS

🤝 B2B

Artificial Intelligence • SaaS • B2B

StackYak is an AI infrastructure orchestration platform that combines inference, GPU compute, and private networking into a single control plane. It provides hosted inference and dedicated GPU capacity, built-in private connectivity, and observability so teams can move models from testing to production without stitching together separate vendors. The platform targets developers, enterprises, and MSPs/resellers, and plans to add storage and security layers in future releases.

📋 Description

• Determine how large language models should run against available GPUs and production latency requirements • Select precision, quantization, tensor parallelism, pipeline parallelism, expert parallelism, memory strategy, batching, concurrency, topology, and serving runtime • Defend, measure, and operate inference architecture decisions • Benchmark, deploy, debug, tune, automate, and operate production inference systems • Own serving runtime selection across vLLM, SGLang, TensorRT-LLM, and successors • Establish concurrency limits, latency and TTFT targets, and throughput commitments • Build benchmarking methodology, harnesses, and reproducibility practices • Manage model loading, startup, health, upgrades, capacity, and failure handling • Operate multi-tenant and dedicated serving systems • Diagnose inference incidents across serving runtimes, GPUs, drivers, hosts, and networks • Turn inference expertise into auditable, tested software and automated deployment decisions • Share technical decisions and participate in production on-call

🎯 Requirements

• Production experience running large language models under real load • Experience serving models across multiple GPUs and multiple nodes • Experience sizing models against hardware • Experience choosing quantization and precision strategies with quality and cost consequences • Experience tuning batching and concurrency • Experience measuring and defending latency, TTFT, throughput, utilization, and cost metrics • Experience in NVIDIA and/or AMD inference environments • Production Python tooling and product development • Linux and systems debugging below the container boundary • Experience finding inference failure root causes outside serving runtimes • Understanding of distributed inference and multi-node networking • Experience benchmarking and comparing serving runtimes • Experience automating deployment decisions, or building schedulers, placement systems, or capacity planners • Ability to operate without narrowly defined ownership or a long ramp-up • Willingness to participate in production and on-call responsibilities

🏖️ Benefits

• Meaningful equity • Remote and distributed work across time zones • Employment or contract engagement depending on location • Direct access to founders • Shared responsibility for production and on-call • Technical decision participation • Flexible scope across inference, infrastructure, and networking

Apply Now

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Similar Jobs

🔥 1 hour ago

Section

11 - 50

📚 Education

🤖 Artificial Intelligence

🤝 B2B

Senior Advisor helping large organizations adopt AI through executive strategy, transformation programs, and Section’s workforce platform. Remote U.S. role with monthly client-site travel.

🇺🇸 United States – Remote

🔥 Funding within the last year

💰 $3.1M Venture Round on 2026-06

⏰ Full Time

🟠 Senior

🤖 Artificial Intelligence

🔥 2 hours ago

Iovance Biotherapeutics, Inc.

501 - 1000

🏥 Healthcare

🧬 Biotechnology

💊 Pharmaceuticals

AI Strategist improving clinical, regulatory, safety, and medical-writing operations for a cancer cell therapy company. Building prototypes, quantifying savings, and guiding AI solutions into production.

🔥 3 hours ago

iteria.us

51 - 200

💼 Consulting

📣 Marketing

📦 Logistics

AI Strategist guiding clients’ AI roadmaps, strategies, and technology integration. Driving AI innovation, training, and business transformation for Iteria.us clients.

🔥 3 hours ago

ICloudLogic

51 - 200

💼 Consulting

🤝 B2B

🎯 Recruiter

Remote AI and machine learning training program for professionals, IT specialists, analysts, and tech enthusiasts. Practical instruction in Python, TensorFlow, Scikit-learn, data analytics, and real-world AI implementation.

🔥 3 hours ago

HCR | Human Centric Recruitment

2 - 10

🎯 Recruiter

💼 Consulting

🤝 B2B

Senior AI Product Manager leading figure-generation and generative media features. Building intuitive AI tools for scientists tackling biomedical and climate challenges.

Developed by Lior Neu-ner. I'd love to hear your feedback — Get in touch via DM or support@remoterocketship.com
Follow Remote Rocketship on LinkedInFollow Remote Rocketship on InstagramFollow Remote Rocketship on FacebookFollow Remote Rocketship on TikTok