Research Lead – Pre-training Safety

🔥 12 hours ago

🌐 United States, Singapore – Remote

🇺🇸 United States – Remote 🇸🇬 Singapore – Remote

🏄 California – Remote

🏄 California – Remote

💵 $290k - $450k / year

⏰ Full Time

🟠 Senior

👻 Ghost score 10%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

FAR.AI

WebsiteLinkedInAll Job Openings

11 - 50 employees

Founded 2022

🤖 Artificial Intelligence

📚 Education

🤝 Non-profit

Artificial Intelligence • Education • Non-profit

FAR. AI is a research and education non-profit organization focused on ensuring advanced artificial intelligence is safe and beneficial for everyone. It conducts technical research on robustness, deception, interpretability, and evaluation of frontier AI systems, runs events and workshops that convene academic and industry leaders, and operates programs (like FAR. Labs and grantmaking) to support researchers and build capacity in trustworthy AI. FAR. AI also publishes findings, provides training and policy-facing convenings, and collaborates with universities and agencies to advance secure, aligned AI.

📋 Description

• Develop and lead FAR.AI’s pre-training safety research workstream • Direct capability-control research focused on removing harmful capabilities while preserving benign ones • Scale methods such as Deep Ignorance to models with over 100B parameters and over 1T tokens • Partner with the red team to stress-test resulting models and analyze scaling to frontier systems • Research improved data filtering using data attribution, influence-based selection, and sophisticated classifiers • Use gradient routing to isolate dual-use capabilities in model components such as MoE experts • Develop training methods to actively remove harmful capabilities through unlearning and next-token prediction • Add synthetic data to pre-training or mid-training to shape model representations and behavior • Build and lead the team and set its research direction • Mentor Members of Technical Staff and remain hands-on by writing code and running experiments • Articulate a research agenda and theory of change for mitigating catastrophic AI risks or increasing beneficial outcomes • Lead novel research projects with potentially unclear progress or success markers • Share findings through academic publications, blog posts, ML conferences, and policymaker briefings • Contribute to FAR.AI’s intellectual environment and research culture • Build a research field through grantmaking and events • Connect research to real-world deployments through independent testing and government advising

🎯 Requirements

• Strong existing research track record in AI or another highly technical subject, such as computer science, mathematics, or physics • Deep experience with language-model pretraining, dataset construction, or controlled training experiments • Experience building large-scale pipelines for scoring, filtering, deduplicating, and sampling training corpora • Strong experimental judgment, including safety–capability evaluations, distribution-shift analysis, and statistically rigorous model comparisons • Ability to build and debug research systems directly, from classifier fine-tuning through distributed training and evaluation • Either a clear research agenda with a theory of change, or a strong track record and research space to develop into an agenda • Experience leading a team, mentoring graduate students, or supporting early-career researchers; informal leadership counts • Ability to communicate novel methods and solutions to technical and non-technical audiences • Substantive engagement with machine learning research through prior research, employment, or sustained independent contribution; not a new entrant • Established AI safety publication record preferred • Comfort writing grant proposals and navigating external collaborations preferred • Full-time availability, 40 hours/week • Ability to work remotely in most countries or in person in Berkeley, California or Singapore

🏖️ Benefits

• Competitive salary of $290,000–$450,000/year depending on experience and location; exceptional candidates may receive a higher salary range • Work-related travel and equipment expenses covered • Catered lunch and dinner at Berkeley offices • Flexible remote or in-person work options • Visa sponsorship for in-person employees • Paid 3–5-day work trial • Sizable compute budgets on a managed cluster • Dedicated engineering team supporting compute infrastructure and experiment scaling • Opportunities to collaborate with governments, leading AI companies, and academics • Professional development through research leadership, mentoring, publications, conferences, grantmaking, and events