Senior Software Engineer, AI Gateway

🕒 October 15

🇺🇸 United States – Remote

💵 $160.9k - $260.7k / year

⏰ Full Time

🟠 Senior

🤖 AI Engineer

👻 Ghost score 1%

What it means: the chance that no one is really hiring for this job.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Docker, Inc

WebsiteLinkedInAll Job Openings

51 - 200 employees

💼 Consulting

☁️ SaaS

💰 $105M Series C on 2022-03

Consulting • SaaS

Docker, Inc is a company that simplifies the lives of developers by tackling the complexity of app development. They offer tools that allow developers to bring their ideas to life through containerization and orchestration, making the development process easier and more efficient. Docker's products, including Docker Desktop and Docker Hub, are widely adopted by millions of developers worldwide, enhancing workflows and providing an integrated development pipeline.

📋 Description

• Design and implement core systems powering the AI Gateway, including the model router, MCP gateway, and control plane • Build infrastructure for dynamic model selection and routing, auto-failover, cost-based routing, and policy enforcement • Own secure credential storage, session summarization, caching, and rate limiting • Develop APIs for OpenAI-compatible interfaces and the Model Context Protocol • Build infrastructure for evaluation, telemetry, replay, and backtesting for agents and LLM workflows • Lead architectural decisions and mentor engineers as the team scales • Tune performance through caching, session summarization, and rate limiting under production load • Contribute to roadmap planning, technical strategy, and cross-functional alignment • Build unified abstractions over model and tool providers • Improve credential and identity handling for tool access • Provide real-time and historical visibility into agent activity with enforceable policies • Maintain low latency and high reliability as traffic scales

🎯 Requirements

• 6+ years of backend engineering experience with production-grade systems • Deep knowledge of distributed and highly scalable systems, cloud-native infrastructure, and API design • Experience building secure, high-throughput services such as gateways, proxies, load balancers, or policy engines • Fluency in Go and/or Rust; both preferred • Familiarity with AI/ML platforms or model serving infrastructure • Strong product mindset focused on developer-facing tools • Ownership mentality with a bias for shipping, learning, and iterating • Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience • Prior experience with OpenAI, Anthropic, or similar LLM APIs • Familiarity with RAG architectures, vector databases, or agent frameworks such as LangChain, AutoGPT, or CrewAI • Experience with policy engines such as OPA, observability frameworks such as OpenTelemetry, or API gateways • Understanding of OAuth2.1, secret management, and cloud IAM systems • Experience with Kubernetes, Docker, and microservices architecture • Candidates must be legally authorized to work in the country where they are applying • Visa sponsorship considered case by case based on business needs

🏖️ Benefits

• Remote-first by design; work from home, with offices in Seattle and Paris for connection and collaboration • Flexible schedule • Generous PTO • Designated quarterly Whaleness Days • Designated end-of-year Whaleness break • Home office support • Technology stipend equivalent to US$100 net per month • Annual learning and development stipend for conferences, courses, certifications, and continued learning • 16 weeks of paid parental leave after six months of employment • Equity for all full-time employees • Medical benefits • Retirement benefits • Paid holidays • Docker swag

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume