Senior Software Engineer, AI Gateway

πŸ•’ October 15

Where this job comes from

Posted on Docker, Inc's careers page β†—Docker, Inc's careers page, not through a job board

First seen by Remote Rocketship October 15

Still open on Docker, Inc's site. Checked 2 hours ago.

Apply Now
Logo of Docker, Inc

Docker, Inc

51 - 200 employees

πŸ’Ό Consulting

☁️ SaaS

πŸ’° $105M Series C on 2022-03

Consulting β€’ SaaS

Docker, Inc is a company that simplifies the lives of developers by tackling the complexity of app development. They offer tools that allow developers to bring their ideas to life through containerization and orchestration, making the development process easier and more efficient. Docker's products, including Docker Desktop and Docker Hub, are widely adopted by millions of developers worldwide, enhancing workflows and providing an integrated development pipeline.

πŸ“‹ Description

β€’ Design and implement core systems powering the AI Gateway, including the model router, MCP gateway, and control plane β€’ Build infrastructure for dynamic model selection and routing, auto-failover, cost-based routing, and policy enforcement β€’ Own secure credential storage, session summarization, caching, and rate limiting β€’ Develop APIs for OpenAI-compatible interfaces and the Model Context Protocol β€’ Build infrastructure for evaluation, telemetry, replay, and backtesting for agents and LLM workflows β€’ Lead architectural decisions and mentor engineers as the team scales β€’ Tune performance through caching, session summarization, and rate limiting under production load β€’ Contribute to roadmap planning, technical strategy, and cross-functional alignment β€’ Build unified abstractions over model and tool providers β€’ Improve credential and identity handling for tool access β€’ Provide real-time and historical visibility into agent activity with enforceable policies β€’ Maintain low latency and high reliability as traffic scales

🎯 Requirements

β€’ 6+ years of backend engineering experience with production-grade systems β€’ Deep knowledge of distributed and highly scalable systems, cloud-native infrastructure, and API design β€’ Experience building secure, high-throughput services such as gateways, proxies, load balancers, or policy engines β€’ Fluency in Go and/or Rust; both preferred β€’ Familiarity with AI/ML platforms or model serving infrastructure β€’ Strong product mindset focused on developer-facing tools β€’ Ownership mentality with a bias for shipping, learning, and iterating β€’ Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience β€’ Prior experience with OpenAI, Anthropic, or similar LLM APIs β€’ Familiarity with RAG architectures, vector databases, or agent frameworks such as LangChain, AutoGPT, or CrewAI β€’ Experience with policy engines such as OPA, observability frameworks such as OpenTelemetry, or API gateways β€’ Understanding of OAuth2.1, secret management, and cloud IAM systems β€’ Experience with Kubernetes, Docker, and microservices architecture β€’ Candidates must be legally authorized to work in the country where they are applying β€’ Visa sponsorship considered case by case based on business needs

πŸ–οΈ Benefits

β€’ Remote-first by design; work from home, with offices in Seattle and Paris for connection and collaboration β€’ Flexible schedule β€’ Generous PTO β€’ Designated quarterly Whaleness Days β€’ Designated end-of-year Whaleness break β€’ Home office support β€’ Technology stipend equivalent to US$100 net per month β€’ Annual learning and development stipend for conferences, courses, certifications, and continued learning β€’ 16 weeks of paid parental leave after six months of employment β€’ Equity for all full-time employees β€’ Medical benefits β€’ Retirement benefits β€’ Paid holidays β€’ Docker swag

Apply Now

πŸ“Š Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Similar Jobs

πŸ•’ October 15

NeuraFlash

201 - 500

πŸ’Ό Consulting

🏭 Manufacturing

πŸ“£ Marketing

Forward Deployed Engineer building rapid Agentforce AI prototypes and client demonstrations. Delivering Salesforce and generative AI solutions for NeuraFlash, an Accenture AI innovation partner.

πŸ•’ October 15

RealPage, Inc.

5001 - 10000

πŸ’Ό Consulting

πŸ“¦ Logistics

πŸ“£ Marketing

Senior Agentic AI Platform Engineer building reusable orchestration, runtime, and reliability infrastructure. Enabling RealPage product teams to develop, release, evaluate, and operate production AI agents.

πŸ‡ΊπŸ‡Έ United States – Remote

πŸ’΅ $159.1k - $270.9k / year

πŸ’° $18.6M Post-IPO Equity - RealPage on 2022-01

⏰ Full Time

🟠 Senior

πŸ€– AI Engineer

πŸ•’ October 15

Basis Technologies

501 - 1000

πŸ’Ό Consulting

πŸ“¦ Logistics

πŸ“£ Marketing

Staff AI Engineer leading Compass, Basis’s LLM-powered agent platform for digital media operations. Architecting agent systems, MCP/A2A integrations, evaluation, and AI-assisted engineering practices.

πŸ•’ October 15

Provectus

501 - 1000

πŸ’Ό Consulting

πŸ₯ Healthcare

🏭 Manufacturing

Senior Forward Deployed AI Engineer building production GenAI systems for enterprise clients. Deploying AWS-based agentic applications across financial services and healthcare.

πŸ•’ October 14

Thomson Reuters

10,000+ employees

πŸ’Ό Consulting

βš–οΈ Legal

πŸ›‘οΈ Insurance

Legal AI Engineer helping Thomson Reuters transform law firm workflows with Legal AI. Designing, testing, and deploying customer solutions across complex legal technology environments.

Built byΒ Lior Neu-ner. I'd love to hear your feedback β€” Get in touch via DM or support@remoterocketship.com
Follow Remote Rocketship on LinkedInFollow Remote Rocketship on InstagramFollow Remote Rocketship on FacebookFollow Remote Rocketship on TikTok