Staff Software Engineer, Inference Routing

Job not on LinkedIn

🔥 5 minutes ago

🇨🇦 Canada – Remote

⏰ Full Time

🟠 Senior

🔴 Lead

🧑‍💻 Full-stack Engineer

Where this job comes from

Posted on PolarGrid's careers page ↗PolarGrid's careers page, not through a job board

First seen by Remote Rocketship 5 minutes ago

Apply Now
Logo of PolarGrid

PolarGrid

2 - 10 employees

🤖 Artificial Intelligence

🔌 API

🤝 B2B

Artificial Intelligence • API • B2B

PolarGrid provides managed AI model inference on dedicated GPUs across a network of sites. Its platform lets businesses deploy custom models through a single endpoint, with traffic routing, scaling, failover, and deployment monitoring.

📋 Description

• Own the routing layer that decides which GPU serves each request • Design and write the router and placement logic • Build, operate, and set the technical direction of the routing system in production • Ensure requests are routed to healthy, available, and sufficiently nearby GPUs • Handle changing capacity and health across multiple sites • Implement safe failure handling so traffic moves transparently when sites or servers go down • Work closely with engineers operating the network and GPU sites • Help hire engineers for the team

🎯 Requirements

• Deep knowledge of how networks, proxies and load balancers work underneath, from building or running production systems • Strong distributed systems fundamentals: consistency, failure handling, retries and backpressure • Production coding experience in a systems language such as Go or Rust • Experience measuring tail latency and identifying its causes • Experience being on call for systems built • Experience setting technical direction that other engineers built on • Clear writing • Effective use of AI tools and ability to check their work • GPU or ML experience is not required

Apply Now

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Similar Jobs

🔥 5 hours ago

StackAdapt

1001 - 5000

📣 Marketing

☁️ SaaS

📱 Media

Staff Software Engineer leading distributed, low-latency inventory systems for StackAdapt’s programmatic advertising platform. Shaping SSP integrations, architecture, performance, and technical direction.

🇨🇦 Canada – Remote

💵 $196k - $269.5k / year

💰 $235M Private Equity Round - StackAdapt on 2025-02

⏰ Full Time

🔴 Lead

🧑‍💻 Full-stack Engineer

🔥 5 hours ago

StackAdapt

1001 - 5000

📣 Marketing

☁️ SaaS

📱 Media

Software Engineer building scalable identity-resolution and backend systems for StackAdapt’s AI-powered advertising platform. Processing high-volume identity data for real-time targeting, audience segmentation, and measurement.

🇨🇦 Canada – Remote

💵 $140k - $192.5k / year

💰 $235M Private Equity Round - StackAdapt on 2025-02

⏰ Full Time

🟡 Mid-level

🟠 Senior

🧑‍💻 Full-stack Engineer

🔥 5 hours ago

Ciena

5001 - 10000

📡 Telecommunications

🔧 Hardware

SME Product Engineer owning modem product engineering and service support strategy. Improving repair solutions, NPI enablement, and lifecycle operations for Ciena networking infrastructure.

🇨🇦 Canada – Remote

💵 $68k - $108.6k / year

💰 Series C on 1995-12

⏰ Full Time

🟠 Senior

🔴 Lead

🧑‍💻 Full-stack Engineer

🔥 6 hours ago

Qualifacts

1001 - 5000

🏥 Healthcare

💼 Consulting

📦 Logistics

Staff Software Engineer building HIPAA-compliant behavioral health patient portals for Qualifacts. Designing distributed backend systems and integrating EHR platforms with Python, AWS, SQL, and React.

🔥 6 hours ago

Tanium

1001 - 5000

💼 Consulting

🏥 Healthcare

📦 Logistics

Senior Software Engineer building cloud-based data services for Tanium’s autonomous IT platform. Developing backend systems with PostgreSQL, cloud storage, APIs, and Kubernetes.

Developed by Lior Neu-ner. I'd love to hear your feedback — Get in touch via DM or support@remoterocketship.com
Follow Remote Rocketship on LinkedInFollow Remote Rocketship on InstagramFollow Remote Rocketship on FacebookFollow Remote Rocketship on TikTok