Staff Software Engineer, AI Gateway

🔥 5 minutes ago

🇺🇸 United States – Remote

💵 $170.3k - $275.6k / year

⏰ Full Time

🔴 Lead

🤖 AI Engineer

👻 Ghost score 2%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Docker, Inc

WebsiteLinkedInAll Job Openings

51 - 200 employees

💼 Consulting

☁️ SaaS

💰 $105M Series C on 2022-03

Consulting • SaaS

Docker, Inc is a company that simplifies the lives of developers by tackling the complexity of app development. They offer tools that allow developers to bring their ideas to life through containerization and orchestration, making the development process easier and more efficient. Docker's products, including Docker Desktop and Docker Hub, are widely adopted by millions of developers worldwide, enhancing workflows and providing an integrated development pipeline.

📋 Description

• Set technical direction for core AI Gateway systems, including the model router, MCP gateway, and control plane • Ensure architectural decisions support 10x usage growth • Identify infrastructure, process, and tooling gaps and drive fixes through production • Define, deliver, and implement the Gateway's technical strategy with engineering leadership • Solve complex problems involving dynamic model selection, auto-failover, cost-based routing, and end-to-end policy enforcement • Serve as the expert for secure credential storage, session summarization, caching, and rate limiting • Build infrastructure for evaluating, replaying, and backtesting agent and LLM behavior • Mentor senior and mid-level engineers through code, design review, and example • Partner with product, design, and leadership on control plane UI direction and product strategy • Define a unified abstraction layer across model and tool providers including OpenAI, Anthropic, Google, and AWS Bedrock • Architect secure, scalable identity and credential vaulting for enterprise-scale access • Build real-time and historical analytics infrastructure for AI agent behavior • Ensure policy enforcement and logging work end-to-end from prompt to tool to response • Set the standard for developer experience through intuitive APIs and first-class observability

🎯 Requirements

• 8+ years of backend engineering experience • Track record of owning production-grade systems at significant scale • Deep specialization in distributed systems, cloud-native infrastructure, and API design • Experience building secure, high-throughput services such as gateways, proxies, load balancers, and policy engines • Fluency in Go and/or Rust • Track record of identifying systemic gaps and driving fixes across team boundaries • Strong product mindset and experience shaping technical direction • Ability to operate autonomously with wide latitude • Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience • Prior experience with OpenAI, Anthropic, or similar LLM APIs is a bonus • Familiarity with RAG architectures, vector databases, or agent frameworks is a bonus • Experience with policy engines, observability frameworks, or API gateways is a bonus • Understanding of OAuth 2.1, secret management, and cloud IAM systems is a bonus • Experience with Kubernetes, Docker, and microservices architecture is a bonus • Sponsorship considered case-by-case based on business needs

🏖️ Benefits

• Equity for all full-time employees • Remote-first work from home • Flexible schedule • Generous PTO • Designated quarterly Whaleness Days • Designated end-of-year Whaleness break • Home office support • Technology stipend equivalent to US$100 net per month • Annual learning and development stipend for conferences, courses, certifications, and continued learning • 16 weeks of paid parental leave after six months of employment • Medical benefits • Retirement benefits • Paid holidays • Docker swag