Senior HPC Cluster Engineer – AI, ML

🔥 19 minutes ago

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

NVIDIA

WebsiteLinkedInAll Job Openings

10,000+ employees

Founded 1993

🏥 Healthcare

🏭 Manufacturing

🤖 Artificial Intelligence

Healthcare • Manufacturing • Artificial Intelligence

NVIDIA is a leading technology company specializing in accelerated computing and artificial intelligence. NVIDIA pioneers advancements in graphical processing units (GPUs), cloud computing, data centers, and virtual reality, with a focus on gaming, automotive, healthcare, and robotics industries. The company's innovations, such as NVIDIA Omniverse, transform traditional digital processes by enabling high-fidelity simulations and rendering tasks. Their applications span various industries, from autonomous vehicles using NVIDIA DRIVE to healthcare solutions with NVIDIA Clara, and AI-driven analytics and workflows.

📋 Description

• Provide leadership in systems administration and service delivery on the AI/HPC fleet by coordinating system upgrades, responding to incidents, and delivering reliability improvements • Collaborate with global teams to deliver a world-class user experience in AI and HPC research • Own day-to-day operations of production AI/HPC clusters, ensuring system health, user satisfaction, and efficient resource utilization • Develop and improve the ecosystem around GPU-accelerated computing, including scalable automation solutions • Build and maintain heterogeneous AI/ML clusters on-premises and in the cloud • Create and cultivate customer and cross-team relationships to meet evolving user needs • Support researchers running workloads, including performance analysis and optimization • Analyze and optimize cluster efficiency, job fragmentation, and GPU waste to meet internal SLA targets • Conduct root cause analysis and suggest corrective actions • Proactively find and fix issues before they occur • Lead SEV triage and postmortems for reliability incidents affecting users or infrastructure • Participate in on-call rotation and incident response for critical production GPU clusters

🎯 Requirements

• Bachelor’s degree in Computer Science, Electrical Engineering or related field or equivalent experience • Minimum 5 years of experience designing and operating large-scale compute infrastructure • Experience with AI/HPC advanced job schedulers such as Slurm, K8s, PBS, RTDA, BCM, or LSF • Proficient in administering CentOS/RHEL and/or Ubuntu Linux distributions • Solid understanding of cluster configuration management tools including BCM, Terraform, Ansible, Puppet, and Salt • Knowledge of container technologies including Docker, Singularity, Podman, Shifter, and Charliecloud • Python programming and Bash scripting • Applied experience with AI/HPC workflows using MPI • Experience analyzing and tuning performance for varied AI/HPC workloads • Passion for continual learning and staying ahead of emerging HPC and AI/ML infrastructure technologies and approaches • Background with NVIDIA GPUs, CUDA programming, NCCL, and MLPerf benchmarking • Experience with AI/ML concepts, algorithms, models, and frameworks such as PyTorch and TensorFlow • Experience with InfiniBand, IPoIB, and RDMA • Understanding of fast, distributed storage systems such as Lustre and GPFS for AI/HPC workloads