Senior AI Inference Engineer

πŸ•’ October 30

Where this job comes from

Posted on StackYak's careers page β†—StackYak's careers page, not through a job board

First seen by Remote Rocketship October 30

Still open on StackYak's site. Checked 1 hour ago.

Apply Now
Logo of StackYak

StackYak

2 - 10 employees

πŸ€– Artificial Intelligence

☁️ SaaS

🀝 B2B

Artificial Intelligence β€’ SaaS β€’ B2B

StackYak is an AI infrastructure orchestration platform that combines inference, GPU compute, and private networking into a single control plane. It provides hosted inference and dedicated GPU capacity, built-in private connectivity, and observability so teams can move models from testing to production without stitching together separate vendors. The platform targets developers, enterprises, and MSPs/resellers, and plans to add storage and security layers in future releases.

πŸ“‹ Description

β€’ Determine model placement, parallelism, precision, and memory strategies across single-GPU, multi-GPU, and multi-node deployments β€’ Select and evaluate serving runtimes including vLLM, SGLang, TensorRT-LLM, and successors β€’ Establish and defend concurrency limits, latency and TTFT targets, and throughput commitments β€’ Develop benchmarking methodology, harnesses, and reproducible measurement practices β€’ Own production model lifecycle including loading, startup, health, upgrades, capacity, and failure handling β€’ Design multi-tenant and dedicated serving approaches β€’ Investigate and resolve inference incidents involving serving runtimes, GPUs, drivers, hosts, and networks β€’ Convert inference expertise into auditable, tested, and automated software β€’ Deploy, benchmark, debug, tune, automate, and operate real inference systems in production β€’ Help automate decisions about hardware fit, precision, model splitting, serving runtime, concurrency, and latency-throughput tradeoffs β€’ Participate in technical decisions and share production/on-call responsibility

🎯 Requirements

β€’ Proven production experience running large language models under real load β€’ Experience serving models across multiple GPUs and multiple nodes β€’ Experience sizing models against hardware β€’ Experience selecting quantization and precision strategies with quality and cost implications β€’ Experience tuning batching and concurrency β€’ Experience measuring and defending latency, TTFT, throughput, utilization, and cost metrics β€’ Experience with NVIDIA and/or AMD inference environments β€’ Production Python development β€’ Linux and systems debugging below the container boundary β€’ Experience diagnosing inference failures outside the serving runtime β€’ Understanding of distributed inference and multi-node networking β€’ Experience benchmarking and comparing serving runtimes β€’ Experience automating deployment decisions, or building schedulers, placement systems, capacity planners, or similar infrastructure β€’ Ability to work across design, implementation, debugging, and operations β€’ Participation in production on-call β€’ Willingness to work in a fast-paced, ambiguous early-stage startup environment

πŸ–οΈ Benefits

β€’ Meaningful equity β€’ Remote and distributed work across time zones β€’ Employment or contract engagement depending on location β€’ Direct access to founders β€’ Shared responsibility for production and on-call β€’ Participation in technical decisions

Apply Now

πŸ“Š Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Similar Jobs

πŸ•’ October 30

Fetch Rewards

501 - 1000

πŸ’Ό Consulting

🍽️ Food & Beverage

πŸ“¦ Logistics

Staff AI Evaluation Lead scaling LLM evaluation, automation, and quality systems at Fetch, a consumer rewards and outcomes-based advertising platform. Defining production standards, metrics, and AI Operations strategy.

πŸ•’ October 30

Function Health

1001 - 5000

πŸ₯ Healthcare

πŸ‘₯ B2C

Senior Clinical Specialist ensuring Function Health’s AI-generated health guidance is clinically accurate, useful, and safe. Partnering with product and engineering teams to improve AI evaluations and member experiences.

πŸ‡ΊπŸ‡Έ United States – Remote

πŸ”₯ Funding within the last year

πŸ’° $450M Debt Financing on 2026-07

⏰ Full Time

🟠 Senior

πŸ€– Artificial Intelligence

πŸ•’ October 30

Clarinet

11 - 50

πŸ’Ό Consulting

🀝 B2B

πŸ“š Education

AI Learning Facilitator delivering adaptive workshops on AI tools, adoption, and behavioral change. Supporting Clarinet’s AI enablement consulting, client relationships, content, and thought leadership.

πŸ•’ October 30

Clarinet

11 - 50

πŸ’Ό Consulting

🀝 B2B

πŸ“š Education

Technical AI Learning Facilitator delivering adaptive workshops for Clarinet, an AI enablement consultancy. Building advanced AI fluency, supporting clients, and shaping rapidly evolving learning content.

πŸ•’ October 30

Tallwave

51 - 200

πŸ’Ό Consulting

πŸ“£ Marketing

AI Systems Configurator building connected AI workflows for Tallwave, a growth partner helping organizations improve operations and business growth. Automating reporting, client intelligence, and delivery systems.

Built byΒ Lior Neu-ner. I'd love to hear your feedback β€” Get in touch via DM or support@remoterocketship.com
Follow Remote Rocketship on LinkedInFollow Remote Rocketship on InstagramFollow Remote Rocketship on FacebookFollow Remote Rocketship on TikTok