Staff HPC Engineer

🕒 August 19

🏄 California, Texas – Remote

🏄 California – Remote 🤠 Texas – Remote

⏰ Full Time

🔴 Lead

👷🏻‍♀️ Engineer

👻 Ghost score 15%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Bitdeer Group

WebsiteLinkedInAll Job Openings

201 - 500 employees

💼 Consulting

📦 Logistics

🏗️ Construction

💰 Post-IPO Equity on 2023-05

Consulting • Logistics • Construction

Bitdeer Group is a leader in the blockchain and high-performance computing industry. It is one of the world’s largest holders and suppliers of hash rate, offering specialized mining infrastructure and high-quality hash rate sharing products. Founded by cryptocurrency pioneer Jihan Wu and led by CEO Matt Linghui Kong, the company is headquartered in Singapore with mining datacenters in the United States, Norway, and Bhutan. Bitdeer is committed to providing comprehensive computing solutions, including cloud services and AI capabilities, while emphasizing dedication, authenticity, and trustworthiness in its mission to be the most reliable provider in the industry.

📋 Description

• Design, deploy, and operate production Slurm clusters on bare metal and virtual machines • Own Slurm cluster architecture, lifecycle, multi-tenant scheduling policy, and reliability across GPU infrastructure • Configure high availability, authentication, topology-aware GPU scheduling, accounting, partitions, QOS, fairshare, preemption, reservations, and tenant TRES limits • Lead adoption of the Slinky slurm-operator and evaluate slurm-bridge for Kubernetes workload co-scheduling • Shift GPU nodes between Slurm batch-training queues and Kubernetes inference capacity using elastic-capacity mechanisms • Operate Pyxis/Enroot and OCI/containerd job paths, supporting MPI/PMIx, module/Spack environments, and customer images • Build passive and active GPU-cluster health checks with automatic drain and job requeue • Own rack burn-in and acceptance testing before paid workloads are scheduled • Deliver reproducible clusters through Terraform, Ansible, golden images, PXE, Redfish, and IPMI • Instrument queue wait time, allocation efficiency, GPU utilization, and job failures through Prometheus/Grafana • Integrate Slurm accounting and GPU-hours with metering and invoicing pipelines • Write runbooks and tenant documentation, onboard and support enterprise customers, handle escalations, and mentor platform engineers

🎯 Requirements

• 8+ years in HPC, systems, or cloud infrastructure engineering • 4+ years operating production Slurm clusters at 100+ GPU-node scale with real users and service-level commitments • Deep hands-on expertise with slurm.conf, gres.conf, topology.conf, cgroup.conf, partitions, QOS, fairshare, preemption, reservations, slurmdbd accounting, slurmrestd, MUNGE/SACK and JWT authentication, and live-cluster version upgrades • Strong GPU and fabric fundamentals, including NVIDIA drivers, Fabric Manager, DCGM, MIG, InfiniBand/RoCEv2, subnet manager/UFM, rail-optimized topology, GPUDirect RDMA, and NCCL tuning and failure diagnosis • Production Kubernetes experience and working knowledge of operator/CRD patterns • Hands-on exposure to at least one Slurm-on-Kubernetes stack: Slinky slurm-operator or slurm-bridge, CoreWeave SUNK, or Nebius Soperator • Experience delivering bare-metal and virtualized compute, including provisioning, firmware/BIOS lifecycle management, KVM/QEMU or public-cloud-equivalent VM clusters, and Terraform/Ansible automation • Working knowledge of parallel and shared storage such as Lustre, GPFS/Spectrum Scale, WEKA, VAST, or NFS • Proficient in Python and Bash for cluster automation • Clear written and verbal communication in English • Go experience is a plus

🏖️ Benefits

• Equal employment opportunities in accordance with country, state, and local laws