Member of Engineering – Pre-training, Data Acquisition

🕒 May 18

Apply Now

Find Similar Remote Jobs

Receive Emails For Remote Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

poolside

WebsiteLinkedInAll Job Openings

51 - 200 employees

Founded 2023

🤖 Artificial Intelligence

🏢 Enterprise

Artificial Intelligence • Enterprise

poolside is a frontier AI lab and enterprise platform that builds and deploys foundation models, multi-agent systems, and developer-facing tools focused on automating complex software work. The company specializes in on-prem and VPC deployments, security-first integrations, governance, and connectors to enterprise data sources so organizations can run agents and models inside their own boundaries. Poolside embeds research and engineering with customers to deliver outcome ownership, risk controls, and measurable business impact while advancing toward AGI by starting in high-consequence software environments.

📋 Description

• Design, build, and operate a large-scale web crawler responsible for acquiring all openly accessible data on the internet • Develop specialized deep crawlers targeting high-value sources to improve recall and coverage • In collaboration with data researchers, own a long-term road map for data acquisition • Build observability, monitoring, and debugging tooling to ensure reliability and transparency across crawl infrastructure • Collaborate with pre-training, post-training, and evaluations teams to align data acquisition priorities with model training needs • Build high-throughput ingestion pipelines for rapidly onboarding partner data and evaluating it for quality

🎯 Requirements

• Strong distributed systems background with proven experience building and operating large-scale infrastructure — data pipelines, web crawlers, or similar • Proficiency in Python, and comfortable optimizing performance and debugging complex systems under production conditions • Hands-on experience with web crawling or large-scale data extraction: understanding of HTTP protocols, distributed job queues, and data parsing at scale • Familiarity with cloud platforms (AWS) and container orchestration (Kubernetes, Docker) for deploying and managing high-throughput workloads • Awareness of the non-technical dimensions of internet-scale crawling: data privacy, robots.txt adherence, and responsible crawl practices • Nice to have: • Prior experience pre-training LLMs • Experience in building trillion-scale SOTA pre-training datasets • Experience translating research to production at scale

🏖️ Benefits

• Fully remote work & flexible hours • 37 days/year of vacation & holidays • 16 weeks of flexible, full-pay parental leave • Health insurance allowance for you & dependents • Company-provided equipment • Well-being, always-be-learning & home office allowances • Frequent team get togethers • Diverse & inclusive people-first culture