Technical Product Manager, Data Science

Job not on LinkedIn

🔥 4 minutes ago

🏄 California – Remote

🏄 California – Remote

💵 $115k - $130k / year

⏰ Full Time

🟡 Mid-level

🟠 Senior

⚒️ Technical Product Manager (TPM)

🦅 H1B Visa Sponsor

This company has sponsored H1B visas in the past.

👻 Ghost score 0%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

CENTRL Inc

WebsiteLinkedInAll Job Openings

51 - 200 employees

💸 Finance

🤖 Artificial Intelligence

📋 Compliance

💰 Series B on 2016-11

Finance • Artificial Intelligence • Compliance

CENTRL Inc. is a company that specializes in providing advanced AI-powered platforms to automate due diligence and risk management processes for the financial services industry. Their solutions include DD360 for investment management, BNM360 for bank network management, and Vendor360 for vendor risk management. The platforms offer automated workflows, centralized data management, advanced analytics, and easy-to-use portals for diverse users such as investors, fund managers, and third-party vendors. CENTRL Inc. aims to enhance efficiency and accuracy in managing due diligence, vendor risks, and compliance through their suite of products, utilizing AI and machine learning technologies to provide actionable insights and streamline complex processes.

📋 Description

• Own the accuracy, reliability, and cost-efficiency of the AI behind CentrlX • Build and own the evaluation foundation, including golden datasets, grading rubrics, LLM-as-judge pipelines calibrated against human labels, and regression suites • Define measurable quality standards for document digitization, extraction, retrieval, groundedness, summaries, responses, evaluations, and multi-step agent runs • Evaluate agent behavior, including tool selection, retrieval quality, step sequencing, and finished deliverable quality • Convert client failures into permanent evaluation cases • Benchmark models across providers on accuracy, latency, and cost • Own model migrations and determine model deployment by workflow • Track AI spend and optimize cost through routing, model tiering, caching, and context strategy • Define logging and tracing requirements for prompts, retrieved context, outputs, tool calls, token counts, and latency • Build in-app feedback capture with Product and Design • Build, label, curate, and maintain evaluation datasets with holdouts and rotation • Serve as the point of contact for AI quality escalations; triage, reproduce, identify root causes, and close the loop • Write user stories and acceptance criteria and implement prompt, configuration, and model changes • Publish regular quality and cost readouts • Work with domain practitioners to encode industry judgment into evaluation rubrics

🎯 Requirements

• Must have work authorization in the USA • 3+ years across product management, data science, or applied AI, including at least 2 years working on LLM-based products in production • Hands-on Python and SQL • Comfortable in a notebook pulling data, running batch inference, and computing metrics • Demonstrated experience building evaluation datasets and harnesses for LLM systems: golden sets, rubric design, LLM-as-judge with human calibration, and regression testing against prompt and model changes • Working fluency with at least one eval or LLM observability platform: Braintrust, LangSmith, Langfuse, Arize Phoenix, W&B Weave, Inspect, Promptfoo, or a comparable in-house harness • Practical understanding of RAG systems: retrieval quality, groundedness and faithfulness, hallucination detection, and chunking and context strategy • Statistical literacy: ability to size a comparison, judge significance, and identify when a difference is not real • Ability to write clear user stories and acceptance criteria and work inside an agile engineering process • Strong written communication • Preferred: experience with document AI, agentic systems, financial services or investment management, inference cost reduction, in-product feedback mechanisms, agent frameworks and MCP • Degree in a quantitative or technical field is preferred