Product Manager – Agent Evaluation Platform

Job not on LinkedIn

November 9

Apply Now

Receive Emails with Similar Jobs

Hyperskill

WebsiteLinkedInAll Job Openings

Education • Artificial Intelligence • SaaS

Hyperskill is reimagining education for the AI era, building intelligent, AI-driven learning systems and productivity tools that adapt to individual learners. It offers project-based courses, career paths, bootcamps, and team training focused on programming (Python, Java, Kotlin, SQL, Go, C++), AI engineering, and product-minded development, plus AI-native tools like Enlighter, Rolloo, InMind Lab, and StoryFlow to help professionals and organizations upskill and deploy practical AI solutions.

11 - 50 employees

📚 Education

🤖 Artificial Intelligence

☁️ SaaS

📋 Description

• As a Product Manager, you'll figure out how to turn agent evaluation research into a real product. • Understanding how companies currently test their AI systems, what they're missing, and how our platform can fill that gap. • Work with our engineering team to build evaluation pipelines that can assess everything from simple chatbots to complex multi-step agent workflows. • Find early customers — companies building AI agents who are frustrated with current testing methods. • Build Product Strategy from Scratch — Define how we position and sell this technology: embeddable SaaS for AI platforms, standalone evaluation service, developer tool integration, or something else. • Execute Go-to-Market — Find first customers among companies building AI agents, build sales processes, and determine pricing models for an emerging evaluation category. • Own Technical Product Vision — Work with engineering on evaluation pipelines, understand testing approaches, and translate assessment methodologies into clear business value.

🎯 Requirements

• Experience with AI-based products or features — you've built AI products or features and faced challenges evaluating their real-world quality and business impact. • You deeply understand AI agents — you've actually built AI agents, used MCPs, understand tool calls, and their security implications. • You understand evaluation methodologies — you've created evals, built testing datasets, and you distinguish between evaluation approaches. • You think in outcomes — you distinguish between 'agent completed the task' and 'agent solved the user's problem effectively'. • You thrive in ambiguity — you'll need to build a lot, figure things out on the go, experiment constantly, and handle multiple different tasks across various areas simultaneously.

🏖️ Benefits

• Contractor agreement with a US-registered legal entity. • 100% remote — work from anywhere in the world • Competitive salary in USD + equity in the product you're working on — we focus on market rates, ready to hear your expectations and prepare an offer matching your expertise • Resources — budget for tools, learning, and whatever you need to succeed • Fast-moving environment — we ship fast, learn fast, and iterate based on real customer feedback