Staff AI Engineer

πŸ•’ October 26

Where this job comes from

Posted on Cobalt's careers page β†—Cobalt's careers page, not through a job board

First seen by Remote Rocketship October 26

Still open on Cobalt's site. Checked 2 hours ago.

Apply Now
Logo of Cobalt

Cobalt

201 - 500 employees

πŸ” Security

☁️ SaaS

Security β€’ SaaS

Cobalt is a leading provider of offensive security services, specializing in modern pentesting solutions. They offer a comprehensive platform that includes dynamic application security testing, network security, and cloud security. Known as the pioneers of Pentest as a Service (PtaaS), Cobalt allows organizations to efficiently scale their security efforts with on-demand access to a pool of expert pentesters. Their services are designed to help businesses identify risks and secure their infrastructure efficiently while ensuring compliance with industry standards. Cobalt's innovative approach and proven experience help companies reduce security risks and improve their overall security posture.

πŸ“‹ Description

β€’ Set technical direction for agent systems across teams, including orchestration, tool use, context and memory management, human oversight and failure recovery β€’ Design, build and operate shared foundations for evaluations, observability, tracing, execution controls and model access with routing and failover β€’ Establish evaluation practices for task completion, accuracy, false positives, security and cost per successful task β€’ Build representative test cases with domain experts and turn production failures into regression tests β€’ Improve resilience to tool failures, timeouts, rate limits and partial execution with limits on retries, runtime and spend β€’ Build scoped permissions, customer data isolation, audit trails and safe tool/code execution with security and platform teams β€’ Establish engineering standards for CI evaluations, prompt/model versioning, regression testing, cost and latency budgets, and staged rollouts β€’ Lead model selection and cost optimization by benchmarking models against real workloads β€’ Partner with teams on production readiness, incident investigation and agent reliability β€’ Help engineering teams use AI coding agents across planning, implementation, review and testing β€’ Mentor engineers and technical leads through design reviews, pairing, internal talks and written guidance β€’ Create a technical roadmap with product, research and security leaders

🎯 Requirements

β€’ Significant software engineering experience, typically 8–10+ years, with a track record of technical leadership across teams β€’ Hands-on experience building and operating LLM-based agents in production β€’ Track record of improving agent reliability through evaluations, non-deterministic failure investigation and measurable success-rate improvements β€’ Strong knowledge of agent architectures, including tool use, orchestration, planning, context and memory management β€’ Experience running LLM workloads through managed cloud model platforms or direct provider APIs, including rate limits, failure handling and cost tracking β€’ Strong backend and distributed systems fundamentals, including APIs, asynchronous workflows, concurrency and fault tolerance β€’ Experience with production observability, incident investigation and safe rollout of system changes β€’ Understanding of security risks in agent systems, including prompt injection, unsafe tool execution and sensitive-data exposure β€’ Evidence of impact beyond individual code, such as shared platforms, improved standards or mentoring engineers β€’ Experience with agent orchestration, tool calling and structured outputs β€’ Experience with Model Context Protocol (MCP), APIs and permission-controlled tool execution β€’ Experience with automated evaluation harnesses, domain-specific test datasets and regression testing in CI β€’ Experience with distributed tracing, structured logs and metrics for multi-step agent workflows β€’ Experience with managed cloud model platforms, direct provider APIs and multi-provider gateways

πŸ–οΈ Benefits

β€’ Competitive compensation and an attractive equity plan β€’ 401(k) program (US) or pension (EU) β€’ Medical, dental, vision and life insurance (US) or statutory healthcare (EU) β€’ Wellness stipend β€’ Work-from-home equipment and Wi-Fi stipend β€’ Learning and development stipend β€’ Flexible, generous paid time off β€’ Paid parental leave β€’ Ongoing mentorship opportunities

Apply Now

πŸ“Š Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Similar Jobs

πŸ•’ October 26

Lunon

2 - 10

πŸ’Ό Consulting

πŸ€– Artificial Intelligence

🀝 B2B

Forward Deployed AI Engineer making ERP products AI-native at Lunon, an AI-native consulting firm. Fine-tuning, evaluating, and serving production LLM systems for clients.

πŸ•’ October 23

8am

501 - 1000

βš–οΈ Legal

☁️ SaaS

πŸ’³ Fintech

Staff Software Engineer building and deploying scalable AI/ML features for 8am’s SaaS platform. Developing Python, LLM, data infrastructure, and cloud deployment solutions for professional services.

πŸ‡ΊπŸ‡Έ United States – Remote

πŸ’΅ $150k - $190k / year

πŸ’° Private equity on 2024-08

⏰ Full Time

πŸ”΄ Lead

πŸ€– AI Engineer

πŸ•’ October 23

VSP Vision Care

1001 - 5000

πŸ₯ Healthcare

πŸ“£ Marketing

🍽️ Food & Beverage

Leading VSP Vision’s enterprise AI platforms, governance, and engineering strategy. Building secure, scalable machine learning, agentic AI, and AI-enabled BI capabilities remotely in the US.

πŸ•’ October 23

Prologis

1001 - 5000

πŸ“¦ Logistics

🏭 Manufacturing

πŸ—οΈ Construction

Technical AI Lead delivering generative AI, RAG, and document intelligence for Prologis’s logistics infrastructure. Improving Leasing, Legal, and document workflows from prototype through production.

πŸ•’ October 23

Everest Technologies, Inc

51 - 200

πŸ’Ό Consulting

πŸ“¦ Logistics

πŸ“£ Marketing

Remote senior C#/.NET developer building APIs and analyzing an enterprise data platform. Later-phase work includes LLM integration, RAG, and GenAI solutions.

Built byΒ Lior Neu-ner. I'd love to hear your feedback β€” Get in touch via DM or support@remoterocketship.com
Follow Remote Rocketship on LinkedInFollow Remote Rocketship on InstagramFollow Remote Rocketship on FacebookFollow Remote Rocketship on TikTok