Research Scientist – Pre-training

🔥 50 minutes ago

🌐 United States, Australia – Remote

🇺🇸 United States – Remote 🇦🇺 Australia – Remote

🏄 California – Remote

🏄 California – Remote

⏰ Full Time

🟡 Mid-level

🟠 Senior

🧬 Research Scientist

👻 Ghost score 25%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Pluralis Research

WebsiteLinkedInAll Job Openings

1 - 10 employees

🤖 Artificial Intelligence

🌐 Web 3

Artificial Intelligence • Web 3

Pluralis Research is a foundational AI research lab focused on Protocol Learning — decentralized, multi‑participant training of foundation models where no single participant holds a full copy of the model. The group develops methods to enable communication‑efficient model and pipeline parallelism, unextractable collaborative models, and high‑compression context parallelism so community‑trained, community‑owned frontier models can scale over low‑bandwidth, internet‑connected devices. Their work targets practical systems and algorithms that make decentralized training competitive with centralized training while enabling new ownership and economic models.

📋 Description

• Build the training system that takes Protocol Learning from an 8B run to frontier scale • Implement and optimize model-parallel distributed pretraining across heterogeneous GPUs • Implement data, pipeline, and tensor parallelism over low-bandwidth, high-latency links • Reduce communication overhead while maintaining model convergence • Build elasticity and fault tolerance so runs survive node churn • Implement robust checkpointing, state synchronization, and recovery as participants join and leave • Build monitoring to show throughput, bottlenecks, and model quality across hundreds of devices • Train and serve large models on heterogeneous hardware in physically different regions over ordinary internet connections

🎯 Requirements

• Hands-on distributed training required: trained models across many devices in PyTorch using FSDP, DeepSpeed, Megatron, or a custom implementation • Understanding of data, tensor, and pipeline parallelism • Production-quality Python • Concurrency and failure handling experience • Profiling before optimizing • Evidence of execution through shipped systems, research code, open-source work, or serious personal projects • Mission alignment with Protocol Learning • Professional-level English proficiency, written and spoken • Nice to have: experience training or serving large language models such as Nemotron, Qwen, or OLMo • Nice to have: P2P networking and NAT traversal experience • Nice to have: post-training and reinforcement learning experience • Nice to have: inference and serving systems experience • Nice to have: experience at proprietary, open-weight, or open-source AI labs

🏖️ Benefits

• Significant ownership/equity for key technical contributors • High base salary • Flexible work environment • Full visa sponsorship and relocation support to either Australia or the US for exceptional candidates • Opportunity to work on open, unsolved research problems • Remote-first culture • Globally distributed team