Inference Optimization Engineer

🔥 17 minutes ago

🌐 United States, Canada – Remote

🇺🇸 United States – Remote 🇨🇦 Canada – Remote

💵 $198k - $286k / year

⏰ Full Time

🟡 Mid-level

🟠 Senior

👷🏻‍♀️ Engineer

🦅 H1B Visa Sponsor

This company has sponsored H1B visas in the past.

👻 Ghost score 0%

What it means: the chance that no one is really hiring for this job.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Modular

WebsiteLinkedInAll Job Openings

51 - 200 employees

🤖 Artificial Intelligence

🏢 Enterprise

💰 $100.5G Series B on 2023-08

Artificial Intelligence • Enterprise

Modular is the next-generation AI developer platform unifying the development and deployment of AI for the world. The company focuses on enabling AI to be accessible to anyone, anywhere, leveraging specialties in machine learning, AI technologies, and platforms such as TensorFlow and PyTorch, along with hardware solutions.

📋 Description

• Build the optimization platform driving inference performance of LLMs served on Modular Cloud across the latest GPU and ASIC architectures • Shape the technical direction of Modular Cloud and deliver LLM performance on the Pareto frontier for agentic use cases • Partner with the GTM team to deliver customized LLM inference tuned to customer use cases • Collaborate across engineering on optimizations spanning GPU kernels through cloud infrastructure • Translate customer engagement insights into technical direction for engineering teams • Publish blog posts on innovative approaches to LLM inference optimization • Travel 2-4 times a year for team events and meetups

🎯 Requirements

• 5+ years of experience in distributed systems or performance engineering • Track record of building durable, reusable software tools and libraries adopted across teams and functions • Sound judgment in evaluating technical tradeoffs and setting priorities • Strong communication and technical leadership skills • Creativity and curiosity in solving complex problems • Collaborative and team oriented mindset • Alignment with Modular’s culture • Candidates must be based in the US or Canada • Helpful but not required: experience with GPU kernel programming, inference engine internals, or distributed inference architectures • Helpful but not required: experience with Kubernetes and cloud native ecosystems • Helpful but not required: familiarity with modern LLM architectures and the latest inference optimization techniques • Must be authorized to work in the U.S. for U.S. employment

🏖️ Benefits

• Comprehensive healthcare coverage • Retirement and savings programs • Employee stock purchase opportunities • Paid time off • Wellbeing resources • Family support programs • Learning and development opportunities • RSU grants • Regular team onsites and local meetups • Traveling 2-4 times a year

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume