Member of Technical Staff – Training Platform

🕒 October 15

🇺🇸 United States – Remote

💵 $150k - $300k / year

⏰ Full Time

🟠 Senior

🔴 Lead

🖥 Software Engineer

Where this job comes from

Posted on Prime Intellect's careers page ↗Prime Intellect's careers page, not through a job board

First seen by Remote Rocketship October 15

Still open on Prime Intellect's site. Checked 21 minutes ago.

Apply Now
Logo of Prime Intellect

Prime Intellect

1 - 10 employees

🤖 Artificial Intelligence

☁️ SaaS

Artificial Intelligence • SaaS • Cloud Computing

Prime Intellect is a company focused on democratizing AI development by providing scalable and decentralized computing resources for training models. Their platform allows users to find and share global compute resources, enabling the training of state-of-the-art models through distributed clusters. They promote the collective ownership of AI innovations, including language and scientific models. Prime Intellect also offers a range of GPU options to facilitate affordable and efficient model training. They aim to advance decentralized training research and open-source AI development on a global scale.

📋 Description

• Build and operate the hosted training platform for LoRA and full fine-tuning runs on managed GPU clusters • Design and operate Kubernetes-based training and inference orchestration across multi-cluster, multi-cloud GPU fleets • Build and maintain Helm charts for reproducible training stacks • Develop Python control-plane agents that monitor pods, report run state, and synchronize clusters • Implement scheduling and autoscaling for heterogeneous H100, H200, and B200 hardware • Operate GitOps workflows through pull requests, Helm values, and CI • Build node-local model caches, checkpoint pipelines, and shared storage for fast cold starts • Operate Prometheus, Grafana, Loki, and DCGM observability infrastructure • Build developer-facing features for job submission, live run monitoring, logs, metrics, model and adapter management, and comparisons • Develop FastAPI backend services and REST APIs connecting the platform to running clusters • Build real-time monitoring and debugging tools, including streaming logs, step-level metrics, and failure analysis • Ship product UI using Next.js, React, TypeScript, shadcn, Tailwind, tRPC, and TanStack Query • Interface with RL trainers, inference servers, and environment servers • Productize new training capabilities, model architectures, RL algorithms, and modes

🎯 Requirements

• Strong working knowledge of open model families and finetuning techniques (LoRA, QLoRA, full FT, RLHF/RLAIF) • Experience with inference engines (vLLM, SGLang, TensorRT-LLM) • Familiarity with GPU hardware tradeoffs (H100, H200, B200, NVLink, interconnects, memory hierarchy) • Understanding of distributed training fundamentals (data, tensor, pipeline, and expert parallelism; NCCL; multi-node scheduling) • Strong Kubernetes operations experience (Helm, CRDs, operators, KEDA, gang scheduling, GPU operator) • Experience debugging production clusters using kubectl, pod lifecycle knowledge, node troubleshooting, and networking • Cloud platform experience; GCP preferred, including GCS, GKE, Cloud Run, and Cloud Tasks • Infrastructure automation experience with Helm, Terraform, or Ansible and a GitOps mindset • Observability experience with Prometheus, Grafana, Loki, OpenTelemetry, and DCGM • Linux fundamentals including networking, namespaces, and performance tuning • Strong Python backend development with FastAPI, async programming, and SQLAlchemy • Experience building Python control-plane agents that interact with Kubernetes APIs • Modern frontend development with TypeScript, React/Next.js, Tailwind, and shadcn • REST and tRPC API design experience • Experience building developer tools, dashboards, and live-monitoring UIs • Experience in platform or infrastructure development; ideally both

🏖️ Benefits

• Significant equity • Flexible work arrangement (remote or San Francisco office) • Full visa sponsorship and relocation support • Professional development budget for courses and conferences • Regular team off-sites and conference attendance • Opportunity to shape the future of decentralized AI development • Contributions to research and open-source work encouraged

Apply Now

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Similar Jobs

🕒 October 15

IPolarity

51 - 200

💼 Consulting

🏥 Healthcare

📦 Logistics

Corticon Developer supporting a Department of Community Health eligibility project. Designing business rules, resolving defects, and validating SNAP, TANF, and Medicaid system updates.

🕒 October 14

DNOW

5001 - 10000

📦 Logistics

🏭 Manufacturing

⚡ Energy

Central Engineering Team Manager leading DNOW’s multidisciplinary engineering team. Overseeing facility design, product development, customer solutions, and process optimization.

🕒 October 14

Rushdown Studios

11 - 50

🎮 Gaming

🤝 B2B

🤖 Artificial Intelligence

Senior Engine Programmer building and optimizing core C++ engine systems for Rushdown Studios’ multiplayer game partners. Solving performance and stability problems across PC and console platforms.

🕒 October 14

Highland Electric Fleets

51 - 200

🚘 Automotive

📦 Logistics

🚗 Transport

Senior Project Developer delivering execution-ready BESS and edge compute projects for Highland Electric Fleets, an Electrification-as-a-Service provider. Managing permitting, utility interconnection, consultants, budgets, and construction handoff.

🕒 October 14

Datadog

1001 - 5000

🔒 Cybersecurity

☁️ SaaS

🏢 Enterprise

Manager leading Datadog’s developer advocacy community programs for its observability and security platform. Developing advocates, customer stories, technical content, and conference talks.

Built by Lior Neu-ner. I'd love to hear your feedback — Get in touch via DM or support@remoterocketship.com
Follow Remote Rocketship on LinkedInFollow Remote Rocketship on InstagramFollow Remote Rocketship on FacebookFollow Remote Rocketship on TikTok