Machine Learning Engineer – Inference Optimization

Job not on LinkedIn

🕒 January 22

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Featherless AI

WebsiteLinkedInAll Job Openings

1 - 10 employees

Founded 2023

🤖 Artificial Intelligence

☁️ SaaS

🔌 API

Artificial Intelligence • SaaS • API

Featherless AI is a serverless AI inference and model hosting provider that offers API access to a large and growing catalog of open-weight models (12,200+), enabling developers and businesses to deploy, fine-tune, and run models at scale without managing servers. The company provides flat subscription pricing with unlimited tokens, GPU orchestration, private/anonymous usage (no logs), and options for enterprise self-hosting or scale units for high concurrency. Featherless AI also operates as an AI research lab focused on open-source and post-transformer model research, claiming significant cost and performance improvements for large models and AI agents.

📋 Description

• Optimize inference latency, throughput, and cost for large-scale ML models in production • Profile and bottleneck GPU/CPU inference pipelines (memory, kernels, batching, IO) • Implement and tune techniques such as: • Quantization (fp16, bf16, int8, fp8) • KV-cache optimization & reuse • Speculative decoding, batching, and streaming • Model pruning or architectural simplifications for inference • Collaborate with research engineers to productionize new model architectures • Build and maintain inference-serving systems (e.g. Triton, custom runtimes, or bespoke stacks) • Benchmark performance across hardware (NVIDIA / AMD GPUs, CPUs) and cloud setups • Improve system reliability, observability, and cost efficiency under real workloads

🎯 Requirements

• Strong experience in ML inference optimization or high-performance ML systems • Solid understanding of deep learning internals (attention, memory layout, compute graphs) • Hands-on experience with PyTorch (or similar) and model deployment • Familiarity with GPU performance tuning (CUDA, ROCm, Triton, or kernel-level optimizations) • Experience scaling inference for real users (not just research benchmarks) • Comfortable working in fast-moving startup environments with ownership and ambiguity • Experience with LLM or long-context model inference • Knowledge of inference frameworks (TensorRT, ONNX Runtime, vLLM, Triton) • Experience optimizing across different hardware vendors • Open-source contributions in ML systems or inference tooling • Background in distributed systems or low-latency services

🏖️ Benefits

• Competitive compensation + meaningful equity at Series A