Senior AI Inference Engineer

🕒 3 days ago

🇺🇸 United States – Remote

⏰ Full Time

🟠 Senior

🤖 Artificial Intelligence

Where this job comes from

Posted on StackYak's careers page ↗StackYak's careers page, not through a job board

First seen by Remote Rocketship 3 days ago

Still open on StackYak's site. Checked 43 minutes ago.

Apply Now
Logo of StackYak

StackYak

2 - 10 employees

🤖 Artificial Intelligence

☁️ SaaS

🤝 B2B

Artificial Intelligence • SaaS • B2B

StackYak is an AI infrastructure orchestration platform that combines inference, GPU compute, and private networking into a single control plane. It provides hosted inference and dedicated GPU capacity, built-in private connectivity, and observability so teams can move models from testing to production without stitching together separate vendors. The platform targets developers, enterprises, and MSPs/resellers, and plans to add storage and security layers in future releases.

📋 Description

• Determine how large language models should run against available GPUs and production latency requirements • Select precision, quantization, tensor parallelism, pipeline parallelism, expert parallelism, memory strategy, batching, concurrency, topology, and serving runtime • Defend, measure, and operate inference architecture decisions • Benchmark, deploy, debug, tune, automate, and operate production inference systems • Own serving runtime selection across vLLM, SGLang, TensorRT-LLM, and successors • Establish concurrency limits, latency and TTFT targets, and throughput commitments • Build benchmarking methodology, harnesses, and reproducibility practices • Manage model loading, startup, health, upgrades, capacity, and failure handling • Operate multi-tenant and dedicated serving systems • Diagnose inference incidents across serving runtimes, GPUs, drivers, hosts, and networks • Turn inference expertise into auditable, tested software and automated deployment decisions • Share technical decisions and participate in production on-call

🎯 Requirements

• Production experience running large language models under real load • Experience serving models across multiple GPUs and multiple nodes • Experience sizing models against hardware • Experience choosing quantization and precision strategies with quality and cost consequences • Experience tuning batching and concurrency • Experience measuring and defending latency, TTFT, throughput, utilization, and cost metrics • Experience in NVIDIA and/or AMD inference environments • Production Python tooling and product development • Linux and systems debugging below the container boundary • Experience finding inference failure root causes outside serving runtimes • Understanding of distributed inference and multi-node networking • Experience benchmarking and comparing serving runtimes • Experience automating deployment decisions, or building schedulers, placement systems, or capacity planners • Ability to operate without narrowly defined ownership or a long ramp-up • Willingness to participate in production and on-call responsibilities

🏖️ Benefits

• Meaningful equity • Remote and distributed work across time zones • Employment or contract engagement depending on location • Direct access to founders • Shared responsibility for production and on-call • Technical decision participation • Flexible scope across inference, infrastructure, and networking

Apply Now

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Similar Jobs

🕒 3 days ago

Section

11 - 50

📚 Education

🤖 Artificial Intelligence

🤝 B2B

Senior Advisor helping large organizations adopt AI through executive strategy, transformation programs, and Section’s workforce platform. Remote U.S. role with monthly client-site travel.

🇺🇸 United States – Remote

🔥 Funding within the last year

💰 $3.1M Venture Round on 2026-06

⏰ Full Time

🟠 Senior

🔴 Lead

🤖 Artificial Intelligence

🕒 3 days ago

Iovance Biotherapeutics, Inc.

501 - 1000

🏥 Healthcare

🧬 Biotechnology

💊 Pharmaceuticals

AI Strategist improving clinical, regulatory, safety, and medical-writing operations for a cancer cell therapy company. Building prototypes, quantifying savings, and guiding AI solutions into production.

🕒 4 days ago

Huron

5001 - 10000

🏥 Healthcare

📦 Logistics

📣 Marketing

AI performance associate measuring enterprise AI adoption, accuracy, and business impact. Optimizing Copilot, ChatGPT Enterprise, Claude for Work, and Gemini deployments at Huron.

🕒 4 days ago

CrowdStrike

5001 - 10000

🔒 Cybersecurity

☁️ SaaS

🤖 Artificial Intelligence

Technical Operations Engineer building and operating production agentic AI systems for CrowdStrike’s cybersecurity platform. Designing secure multi-agent solutions and handling incident response.

🕒 4 days ago

Smartcat

51 - 200

🤖 Artificial Intelligence

☁️ SaaS

AI Strategist building custom AI workflows and agents for Smartcat’s enterprise localization platform. Translating strategic customers’ business problems into scalable, measurable solutions.

Developed by Lior Neu-ner. I'd love to hear your feedback — Get in touch via DM or support@remoterocketship.com
Follow Remote Rocketship on LinkedInFollow Remote Rocketship on InstagramFollow Remote Rocketship on FacebookFollow Remote Rocketship on TikTok