Senior AI Product Manager – Evals & Annotations

Job not on LinkedIn

πŸ•’ November 10

Where this job comes from

Posted on Arize AI's careers page β†—Arize AI's careers page, not through a job board

First seen by Remote Rocketship November 10

Still open on Arize AI's site. Checked 42 minutes ago.

Apply Now
Logo of Arize AI

Arize AI

51 - 200 employees

Founded 2019

πŸ€– Artificial Intelligence

☁️ SaaS

🏒 Enterprise

Artificial Intelligence β€’ SaaS β€’ Enterprise

Arize AI is a company that provides a comprehensive AI observability and evaluation platform designed for AI engineers to build and manage AI applications, particularly those involving large language models (LLMs). The platform offers end-to-end AI tracing, monitoring, and evaluation tools to help users identify and resolve performance bottlenecks, ensure data quality, and improve the performance of AI systems. Arize also supports experimentation and visualization, enabling users to iterate rapidly on AI applications and maintain high-quality service delivery. The company caters to top AI companies, offering solutions for both online and offline LLM evaluation and observability. Arize's tools are particularly valuable for AI model monitoring, troubleshooting, and performance optimization, helping businesses mitigate risks and improve AI outcomes.

πŸ“‹ Description

β€’ Own Arize's Evals & Annotations product area end-to-end, including eval creation and execution, human review, annotation workflows, and the broader evaluation lifecycle β€’ Set product strategy and roadmap, balancing customer needs, platform investments, UX improvements, and emerging AI and agent-development opportunities β€’ Develop strong workflow opinions, independently prototype solutions, and partner with Design on intuitive, polished experiences β€’ Shape the future of evaluations for autonomous and managed agents, including opportunities within the broader Dynatrace ecosystem β€’ Work with Engineering on evaluation execution, backend systems, orchestration, and data infrastructure β€’ Contribute to technical discussions, implementation approaches, and tradeoffs β€’ Have multiple customer conversations each week to understand evaluation practices, uncover problems, and validate product direction β€’ Collaborate with cofounders and cross-functional teams on product direction, design exploration, and prioritization β€’ Make prioritization decisions and drive products from problem definition through launch, adoption, and iteration

🎯 Requirements

β€’ Significant product management experience independently owning technically complex developer-facing products; exact years flexible for candidates with exceptional technical depth, product judgment, and ownership β€’ Strong understanding of LLM evaluations and AI development workflows, including automated evals, human feedback, experimentation, and iteration on AI applications and agents β€’ Experience with developer tools, AI infrastructure, ML platforms, or adjacent technical products β€’ Solid understanding of backend systems and software architecture β€’ Strong technical depth and hands-on engineering ability β€’ Ability to debug issues independently and navigate code when needed β€’ Ability to put up small fixes or PRs β€’ Ability to build prototypes and technical proofs of concept β€’ Ability to propose technical implementations and reason through tradeoffs β€’ Ability to hold credible technical discussions with engineers β€’ Excellent product and UX instincts; ability to define and prototype core product experiences β€’ Strong ownership, initiative, and bias toward action β€’ Ability to navigate ambiguity and make tradeoffs β€’ Collaborative, low-ego approach to working with founders, engineers, and designers β€’ Experience working closely with technical customers β€’ Strong communication skills and ability to move between strategy and execution

πŸ–οΈ Benefits

β€’ Competitive equity package β€’ Medical insurance β€’ Dental insurance β€’ Vision insurance β€’ 401(k) plan β€’ Unlimited paid time off β€’ Generous parental leave plan β€’ Mental and wellness support β€’ WFH monthly stipend for co-working spaces β€’ Optional in-person work in New York City and the San Francisco Bay Area β€’ Regular chats with industry experts, researchers, and ethicists β€’ Culturally conscious events such as LGBTQ trivia during pride month β€’ Active Lady Arizers subgroup

Apply Now

πŸ“Š Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Similar Jobs

πŸ•’ November 10

DoorDash

10,000+ employees

🍽️ Food & Beverage

πŸ“¦ Logistics

πŸ›οΈ eCommerce

Staff Product Manager owning HR partner products and workflow automation for DoorDash, Deliveroo, and Wolt. Improving People-team cost, quality, speed, and employee support experiences.

πŸ•’ November 10

LawnStarter

51 - 200

πŸ‘₯ B2C

πŸͺ Marketplace

🏠 Real Estate

Lead Product Manager owning personalized consumer experiences for LawnStarter’s on-demand lawn and home-services marketplace. Driving retention, cross-sell, scheduling, and responsible AI product strategy.

πŸ•’ November 10

Dave

201 - 500

πŸ’³ Fintech

🏦 Banking

πŸ‘₯ B2C

Staff Product Manager owning Dave’s loan management system and lending infrastructure. Building scalable, compliant credit products for Americans underserved by traditional financial institutions.

πŸ•’ November 10

Instacart

1001 - 5000

🍽️ Food & Beverage

πŸ“¦ Logistics

πŸ›οΈ eCommerce

Senior Product Manager owning acquisition, retention, and expansion for Instacart Business. Driving growth across business grocery delivery features, retailer sites, and partnerships.

πŸ‡ΊπŸ‡Έ United States – Remote

πŸ’΅ $158k - $199.5k / year

πŸ’° $232M Venture Round on 2021-11

⏰ Full Time

🟑 Mid-level

🟠 Senior

βœ… Product Manager

πŸ•’ November 10

OPENLANE

1001 - 5000

πŸ“¦ Logistics

πŸ’Ό Consulting

πŸ“£ Marketing

Principal Product Manager shaping data products, APIs, machine-learning models, and fraud solutions for OPENLANE’s digital used-vehicle marketplace. Driving monetization, product strategy, and cross-functional execution.

Built byΒ Lior Neu-ner. I'd love to hear your feedback β€” Get in touch via DM or support@remoterocketship.com
Follow Remote Rocketship on LinkedInFollow Remote Rocketship on InstagramFollow Remote Rocketship on FacebookFollow Remote Rocketship on TikTok