Senior/Staff Software Engineer, Inference, Compute Infrastructure Engineering

🔥 14 hours ago

🇮🇳 India – Remote

⏰ Full Time

🟠 Senior

👷 Infrastructure Engineer

👻 Ghost score 12%

What it means: the chance that no one is really hiring for this job.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Together AI

WebsiteLinkedInAll Job Openings

11 - 50 employees

🤖 Artificial Intelligence

☁️ SaaS

🏢 Enterprise

💰 $102.5M Series A on 2023-11

Artificial Intelligence • SaaS • Enterprise

Together AI is a company focused on accelerating the generative AI lifecycle by providing solutions for training, fine-tuning, and deploying AI models. They offer an end-to-end platform for leveraging pre-trained models, adapting them to specific needs, or building custom models from scratch. Together AI supports AI compute solutions with its Together Inference engine and Together GPU Clusters, allowing for scalable and efficient AI operations. Their infrastructure is powered by advanced GPUs and optimized software stacks for high performance in AI tasks.

📋 Description

• Build the provisioning state machine modeling the full physical-host lifecycle from discovery and inference bring-up through GPU driver/CUDA installation, health validation, and decommission/RMA. • Build declarative self-service APIs and a control plane for requesting, scaling, and tearing down inference clusters. • Automate self-healing by detecting degraded or failed nodes, draining them safely, triggering repair or replacement, and restoring healthy capacity. • Own pipeline reliability through idempotency, retries, rollback, and drift detection. • Partner with the inference/ML platform team to encode cluster topology, interconnect, and scheduling constraints as platform abstractions. • Apply strong typing, automated tests, code review, versioning, and CI/CD to infrastructure code. • Build Kubernetes-native controllers, reconciliation loops, API/CLI self-service surfaces, and event-driven health, remediation, and utilization systems for an efficient GPU inference fleet. • Operate and support the software in production.

🎯 Requirements

• Strong software engineering background in Go, Python, Rust, or similar — you write and test real software for a living. • Experience with durable workflow orchestration tools such as Temporal, Cadence, or equivalent to run long-lived, manifest-driven workflows that survive failures and resume mid-execution. • Experience building software control planes or orchestration systems that model state and reconcile it over time (e.g., Kubernetes controllers/operators, custom reconciliation loops, workflow engines). • Experience with event-driven systems — designing and building software around message queues, event streams, or pub/sub (e.g., Kafka, NATS, SQS) rather than polling or cron-driven scripts. • A product mindset; experience building internal platforms or APIs consumed by other engineering teams and caring about developer experience. • Nice to have: Exposure to bare-metal provisioning (PXE/iPXE, Redfish/IPMI, BMC), networking fundamentals (VLANs, BGP, fabric design), or GPU/accelerator infrastructure. • Nice to have: Experience with GPU cluster software stacks (NCCL, CUDA, InfiniBand/RoCE). • Nice to have: Prior work at a hyperscaler, GPU cloud, or datacenter-scale infrastructure organization. • Nice to have: Systems programming in Rust or Go.

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume