Member of Engineering – Pre-training, Data Acquisition

🕒 Maio 18

Candidatar-se

Encontrar Vagas Remotas Similares

Receber Emails de Vagas Remotas

📊 Verifique sua pontuação de currículo para esta vaga

Melhore suas chances de conseguir uma entrevista verificando sua pontuação de currículo antes de se candidatar.

Enviar Currículo

poolside

SiteLinkedInTodas as Vagas

51 - 200 funcionários

Fundada em 2023

🤖 Inteligência Artificial

🏢 Corporativo

Artificial Intelligence • Enterprise

Poolside é uma aceleradora projetada especificamente para fundadores e builders de Web3. Ela oferece suporte a projetos de finanças descentralizadas (DeFi), games, governança, infraestrutura e NFTs. Com um ecossistema robusto de 20. 000 membros — incluindo mentores, investidores e builders de Web3 — a Poolside co-lançou e apoiou mais de 110 projetos. A aceleradora proporciona acesso diferenciado a mentoria e expertise técnica para ajudar projetos Web3 a escalar e alcançar lançamentos bem-sucedidos. A Poolside também se engaja com empresas e protocolos líderes para impulsionar o crescimento e a inovação no espaço Web3.

Descrição

• Design, build, and operate a large-scale web crawler responsible for acquiring all openly accessible data on the internet • Develop specialized deep crawlers targeting high-value sources to improve recall and coverage • In collaboration with data researchers, own a long-term road map for data acquisition • Build observability, monitoring, and debugging tooling to ensure reliability and transparency across crawl infrastructure • Collaborate with pre-training, post-training, and evaluations teams to align data acquisition priorities with model training needs • Build high-throughput ingestion pipelines for rapidly onboarding partner data and evaluating it for quality

🎯 Requisitos

• Strong distributed systems background with proven experience building and operating large-scale infrastructure — data pipelines, web crawlers, or similar • Proficiency in Python, and comfortable optimizing performance and debugging complex systems under production conditions • Hands-on experience with web crawling or large-scale data extraction: understanding of HTTP protocols, distributed job queues, and data parsing at scale • Familiarity with cloud platforms (AWS) and container orchestration (Kubernetes, Docker) for deploying and managing high-throughput workloads • Awareness of the non-technical dimensions of internet-scale crawling: data privacy, robots.txt adherence, and responsible crawl practices • Nice to have: • Prior experience pre-training LLMs • Experience in building trillion-scale SOTA pre-training datasets • Experience translating research to production at scale

🏖️ Benefícios

• Fully remote work & flexible hours • 37 days/year of vacation & holidays • 16 weeks of flexible, full-pay parental leave • Health insurance allowance for you & dependents • Company-provided equipment • Well-being, always-be-learning & home office allowances • Frequent team get togethers • Diverse & inclusive people-first culture