Staff Engineer, Inference Optimizations

🔥 2 minutes ago

🍂 Massachusetts – Remote

🍂 Massachusetts – Remote

💵 $191.2k - $239k / year

⏰ Full Time

🔴 Lead

🧑‍💻 Full-stack Engineer

🦅 H1B Visa Sponsor

This company has sponsored H1B visas in the past.

👻 Ghost score 0%

What it means: the chance that no one is really hiring for this job.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

DigitalOcean

WebsiteLinkedInAll Job Openings

1001 - 5000 employees

Founded 2011

☁️ SaaS

SaaS • Cloud Computing

DigitalOcean is a cloud infrastructure provider that offers a suite of products and services for developers to build, deploy, and scale applications. Their platform provides comprehensive tutorials, reference material, and support documentation to assist users in managing resources effectively using their API and CLI tools. With features like Droplets (virtual machines), managed databases, Kubernetes, and a marketplace for third-party applications, DigitalOcean focuses on simplicity and performance. They cater to both individual developers and larger organizations looking for cloud solutions that are easy to implement and manage.

📋 Description

• Lead technical strategy for benchmarking and performance optimization at inference engine and GPU kernel layers • Optimize attention layers, memory and precision management, and parallelization across multi-node GPU clusters • Improve batch-size performance using AMD’s AITER library for AMD MI355X and tune CK/ASK kernels for FP8/BF16 • Identify kernel-fusion opportunities for GLM-5 Transformer-block kernels, including FlashAttention and RMS Norm • Tune expert gateway router kernels for MoE models such as Qwen3-235B, DeepSeek V3, and GLM-5 • Serve as subject matter expert on NVIDIA and AMD GPUs and software stacks including CUDA, ROCm, TensorRT, and OpenAI Triton • Develop and deploy quantization techniques including FP8, INT8, and FP4 • Lead code and design reviews and mentor engineers through influence • Partner with Product Management and TPMs to translate hardware limits into shippable product features • Contribute to and integrate open-source AI technologies and participate in GPU infrastructure and model performance communities • Guide the technical roadmap for DigitalOcean’s high-performance inference fleet

🎯 Requirements

• 5+ years of experience in high-performance computing or AI infrastructure • Proven track record of solving compute utilization and memory bandwidth bottlenecks • Deep familiarity with the Gen AI landscape, including LLM, VLM, and LMM model families • Hands-on experience with attention-layer optimizations and parallelization across distributed GPU environments • Comprehensive understanding of NVIDIA and AMD GPU architectures and software ecosystems, including CUDA and ROCm • Extensive experience integrating, building with, and contributing to open-source software projects • Excellent system design skills related to low-level GPU programming, optimization, memory access patterns, and parallel execution • Experience acting as a technical lead and driving design and delivery through cross-functional alignment • Deep understanding of GPU architectures, including SMs, warp scheduling, and Tensor Cores • Expert-level Triton or CUDA proficiency • Expert-level GPU kernel development; custom CUDA kernels or Triton compiler contributions desired • Ability to work remotely

🏖️ Benefits

• Reimbursement for relevant conferences, training, and education, subject to company policy • Flexible time off • Local team meetups • Mental health and well-being support • Annual bonus plan eligibility for non-sales roles • Possible new hire equity grant, subject to plan terms and local eligibility • Employee Stock Purchase Program in participating countries • Comprehensive total rewards package

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume