Senior Networking Solution Test Engineer – AI Cluster Debugging

🕒 March 3

Apply Now
Find Similar Remote Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Logo of NVIDIA

NVIDIA

10,000+ employees

Founded 1993

🤖 Artificial Intelligence

🎮 Gaming

Artificial Intelligence • Gaming • Automotive

NVIDIA is a leading technology company specializing in accelerated computing and artificial intelligence. NVIDIA pioneers advancements in graphical processing units (GPUs), cloud computing, data centers, and virtual reality, with a focus on gaming, automotive, healthcare, and robotics industries. The company's innovations, such as NVIDIA Omniverse, transform traditional digital processes by enabling high-fidelity simulations and rendering tasks. Their applications span various industries, from autonomous vehicles using NVIDIA DRIVE to healthcare solutions with NVIDIA Clara, and AI-driven analytics and workflows.

📋 Description

• Design and review test and product requirements across the NVLink, Ethernet and InfiniBand / NIC / DPU / Switch portfolio, focusing on large‑scale AI cluster behavior • Build and maintain realistic customer‑like testbeds, including heterogeneous hardware, OS / driver combinations and complex network fabrics • Own end‑to‑end cluster troubleshooting: reproduce customer scenarios, triage across the stack and drive issues to root cause and fix • Read and understand relevant source code to identify defects, validate fixes and improve logging and instrumentation • Collaborate closely with development teams to debug NCCL, RoCE/RDMA and related networking components using logs, code inspection and targeted experiments • Define tests and guide the automation team to implement robust, debuggable suites that produce actionable logs, metrics and traces • Run Regression, Performance, Functional and Scale testing, analyze results and provide clear, data‑driven reports to collaborators • Profile and benchmark deep learning training and inference workloads, correlating model‑level metrics with system and network telemetry to uncover bottlenecks

🎯 Requirements

• B.A./B.Sc. in Computer Science, Electrical Engineering, or equivalent IT/Network/Systems experience • 8+ years of hands‑on networking or system‑level testing and debugging on Linux • Strong Linux networking and debugging skills (for example perf, tcpdump, ethtool, iproute2) • Proven production‑grade debugging experience: forming hypotheses, running experiments, and driving issues to root cause under pressure • Expertise in host‑side NIC validation and tuning (offloads, queues, interrupts, firmware/driver interactions) • Strong knowledge of AI networking libraries (such as NCCL) and protocols (such as RoCE and RDMA), including performance and correctness debugging • Ability to read and reason about source code (C/C++/Python or similar) and collaborate closely with developers on fixes • Solid scripting and automation skills with Bash / Python / Ansible for setup, log collection, and experiment orchestration • Fast learner, familiar with modern AI tools and workflows, able to adapt quickly • Excellent analytical, problem‑solving and communication skills, with strong ownership and a collaborative approach

🏖️ Benefits

• Reasonable accommodations to ensure all individuals can participate in the job application or interview process • Health insurance • Retirement plans • Paid time off

Apply Now