Technical Product Manager, Data Science

Job not on LinkedIn

πŸ•’ September 21

Where this job comes from

Posted on CENTRL Inc's careers page β†—CENTRL Inc's careers page, not through a job board

First seen by Remote Rocketship September 21

Still open on CENTRL Inc's site. Checked 5 hours ago.

Apply Now
Logo of CENTRL Inc

CENTRL Inc

51 - 200 employees

πŸ’Έ Finance

πŸ€– Artificial Intelligence

πŸ“‹ Compliance

πŸ’° Series B on 2016-11

Finance β€’ Artificial Intelligence β€’ Compliance

CENTRL Inc. is a company that specializes in providing advanced AI-powered platforms to automate due diligence and risk management processes for the financial services industry. Their solutions include DD360 for investment management, BNM360 for bank network management, and Vendor360 for vendor risk management. The platforms offer automated workflows, centralized data management, advanced analytics, and easy-to-use portals for diverse users such as investors, fund managers, and third-party vendors. CENTRL Inc. aims to enhance efficiency and accuracy in managing due diligence, vendor risks, and compliance through their suite of products, utilizing AI and machine learning technologies to provide actionable insights and streamline complex processes.

πŸ“‹ Description

β€’ Own the accuracy, reliability, and cost-efficiency of the AI behind CentrlX β€’ Build and own the evaluation foundation, including golden datasets, grading rubrics, LLM-as-judge pipelines calibrated against human labels, and regression suites β€’ Define measurable quality standards for document digitization, extraction, retrieval, groundedness, summaries, responses, evaluations, and multi-step agent runs β€’ Evaluate agent behavior, including tool selection, retrieval quality, step sequencing, and finished deliverable quality β€’ Convert client failures into permanent evaluation cases β€’ Benchmark models across providers on accuracy, latency, and cost β€’ Own model migrations and determine model deployment by workflow β€’ Track AI spend and optimize cost through routing, model tiering, caching, and context strategy β€’ Define logging and tracing requirements for prompts, retrieved context, outputs, tool calls, token counts, and latency β€’ Build in-app feedback capture with Product and Design β€’ Build, label, curate, and maintain evaluation datasets with holdouts and rotation β€’ Serve as the point of contact for AI quality escalations; triage, reproduce, identify root causes, and close the loop β€’ Write user stories and acceptance criteria and implement prompt, configuration, and model changes β€’ Publish regular quality and cost readouts β€’ Work with domain practitioners to encode industry judgment into evaluation rubrics

🎯 Requirements

β€’ Must have work authorization in the USA β€’ 3+ years across product management, data science, or applied AI, including at least 2 years working on LLM-based products in production β€’ Hands-on Python and SQL β€’ Comfortable in a notebook pulling data, running batch inference, and computing metrics β€’ Demonstrated experience building evaluation datasets and harnesses for LLM systems: golden sets, rubric design, LLM-as-judge with human calibration, and regression testing against prompt and model changes β€’ Working fluency with at least one eval or LLM observability platform: Braintrust, LangSmith, Langfuse, Arize Phoenix, W&B Weave, Inspect, Promptfoo, or a comparable in-house harness β€’ Practical understanding of RAG systems: retrieval quality, groundedness and faithfulness, hallucination detection, and chunking and context strategy β€’ Statistical literacy: ability to size a comparison, judge significance, and identify when a difference is not real β€’ Ability to write clear user stories and acceptance criteria and work inside an agile engineering process β€’ Strong written communication β€’ Preferred: experience with document AI, agentic systems, financial services or investment management, inference cost reduction, in-product feedback mechanisms, agent frameworks and MCP β€’ Degree in a quantitative or technical field is preferred

Apply Now

πŸ“Š Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Similar Jobs

πŸ•’ September 21

CreatorIQ

201 - 500

πŸ“£ Marketing

πŸ“± Media

☁️ SaaS

Technical Product Manager unifying creator-platform metrics for CreatorIQ’s influencer marketing software. Defining measurement logic, validating APIs and data, and shaping analytics products customers trust.

πŸ•’ September 17

Wave Mobile Money

501 - 1000

πŸ’Ό Consulting

πŸ“¦ Logistics

πŸ’³ Fintech

Technical Product Manager improving developer experience, tooling, and AI workflows at Wave. Leading DevEx engineers for a mobile financial-services company serving Africa.

πŸ•’ September 16

Keyfactor

201 - 500

🚘 Automotive

πŸ₯ Healthcare

πŸ” Security

Technical Product Manager owning AgileSec product capabilities at Keyfactor, a provider of cryptographic identity and trust infrastructure. Driving enterprise security workflows, launches, and customer outcomes.

πŸ•’ September 15

Mirantis

501 - 1000

πŸ’Ό Consulting

πŸ₯ Healthcare

πŸ“¦ Logistics

Technical Product Manager owning observability strategy for Mirantis’ k0rdent AI GPU infrastructure control plane. Defining telemetry, monitoring, and roadmap priorities for distributed AI workloads.

πŸ•’ September 10

Emergent Software

51 - 200

πŸ’Ό Consulting

πŸ₯ Healthcare

☁️ SaaS

Technical Product Owner delivering cloud data, analytics, and BI solutions for Emergent Software’s clients. Managing requirements, backlogs, agile delivery, and stakeholder communication remotely.

Built byΒ Lior Neu-ner. I'd love to hear your feedback β€” Get in touch via DM or support@remoterocketship.com
Follow Remote Rocketship on LinkedInFollow Remote Rocketship on InstagramFollow Remote Rocketship on FacebookFollow Remote Rocketship on TikTok