Principal Engineer – Data Retrieval

🔥 13 hours ago

🇨🇭 Switzerland – Remote

⏰ Full Time

🔴 Lead

🚰 Data Engineer

👻 Ghost score 11%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

NVIDIA

WebsiteLinkedInAll Job Openings

10,000+ employees

Founded 1993

🏥 Healthcare

🏭 Manufacturing

🤖 Artificial Intelligence

Healthcare • Manufacturing • Artificial Intelligence

NVIDIA is a leading technology company specializing in accelerated computing and artificial intelligence. NVIDIA pioneers advancements in graphical processing units (GPUs), cloud computing, data centers, and virtual reality, with a focus on gaming, automotive, healthcare, and robotics industries. The company's innovations, such as NVIDIA Omniverse, transform traditional digital processes by enabling high-fidelity simulations and rendering tasks. Their applications span various industries, from autonomous vehicles using NVIDIA DRIVE to healthcare solutions with NVIDIA Clara, and AI-driven analytics and workflows.

📋 Description

• Architect and scale data acquisition infrastructure powering NVIDIA’s foundation models, initially for code and STEM data for Nemotron. • Design and own a unified platform for source discovery, URL indexing, crawl frontiers, predictive ranking, scheduling, distributed fetching, and ingestion. • Design scalable frontier storage and queueing systems with high-throughput writes, efficient lookups, deduplication, and quality-, diversity-, and freshness-based scheduling. • Build a continuous discovery loop that extracts links and feeds new candidates into the crawl frontier. • Develop a shared crawl execution layer that fetches reliably and politely at scale, including dynamic websites. • Build extensible pipelines for open-web, licensed, and partner content, including text, PDFs, code repositories, and multimodal assets. • Define stable source-connector interfaces. • Partner with parsing and curation teams to preserve code, math, and structured content and improve accepted data yield. • Implement provenance and policy enforcement with legal, privacy, security, and data governance partners. • Translate publisher preferences, opt-outs, licensing terms, and use restrictions into auditable downstream controls. • Own production readiness, capacity planning, observability, fault tolerance, recovery, and cost efficiency. • Instrument the pipeline end to end and set service objectives for throughput, freshness, reliability, and training cutoffs. • Work with research and product teams to evaluate internal crawls and external datasets through comparable pilots. • Deliver reproducible data releases connecting source coverage and processing quality to controlled training experiments. • Coordinate cross-team dependencies and tradeoffs between immediate data delivery and long-term platform development. • Lead architecture reviews, critical implementation work, engineer mentoring, and team building.

🎯 Requirements

• A BS, MS, or PhD in Computer Science, Computer Engineering, or a related technical field, or equivalent experience. • 15+ years of relevant experience in software engineering, including designing and operating large-scale distributed systems, setting technical direction, and delivering complex infrastructure across multiple teams. • Deep experience with web crawling or closely related discovery, indexing, or ingestion infrastructure, including distributed scheduling, stateful services, storage, and high-throughput data processing. • Strong hands-on engineering and system design skills. • Ability to move between architecture, implementation, performance analysis, and production debugging. • Experience building batch or streaming pipelines and reasoning about consistency, retries, duplicate processing, backpressure, and recovery. • Track record of improving throughput, latency, resource efficiency, or operational reliability. • Experience delivering data platforms for research, ML, search, or other demanding technical users. • Ability to translate evolving requirements into clear interfaces and measurable engineering outcomes. • Strong technical communication and leadership through influence across engineering, product, research, and governance teams. • Experience architecting a web-wide crawl platform, scaling crawl frontier storage, or consolidating multiple acquisition systems into shared infrastructure. • Direct experience supplying foundation-model pretraining data, including targeted discovery, quality-aware crawl ranking, and acquisition of code or technical content. • Experience implementing publisher consent, training-use opt-outs, dataset licensing controls, or provenance infrastructure at scale. • Background with browser-based or agentic data collection, programmable extraction, and sandboxed environments for tool-use data or post-training evaluation. • Experience onboarding licensed or multimodal datasets, building reusable partner integrations, or delivering streaming feeds with low latency for retrieval and grounding.