Product Data Scientist – AI Evaluation, Quality

🔥 27 minutes ago

🇪🇸 Spain – Remote

⏰ Full Time

🟡 Mid-level

🟠 Senior

📊 Data Scientist

👻 Ghost score 16%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Finom

WebsiteLinkedInAll Job Openings

501 - 1000 employees

Founded 2019

💳 Fintech

💸 Finance

🤝 B2B

Fintech • Finance • B2B

Finom is a fintech company offering comprehensive financial services for SMEs, freelancers, and companies. The platform provides tools for invoicing, expense management, and easy online account opening with added benefits like cashback and international IBAN accounts. Finom ensures data security through partnerships with banks like SolarisBank and Treezor, making it a fast and reliable service for business financial management.

📋 Description

• Join the AI Team driving all AI products and technology at Finom • Own and extend the offline evaluation suite across products, including capability and regression datasets, judges, and metrics • Build and maintain online quality dashboards covering resolution rate, CSAT, thumbs up/down, LLM-as-judge signals, error rate, and latency • Close the production feedback loop by mining failure patterns from real traffic, turning them into regression cases, and proposing fixes to Product and domain experts • Harden evaluation methodology, including judge stability and non-determinism handling • Translate metrics into decisions through weekly syncs and clear trade-offs • Own the evaluation loop across AI products, including pre-launch quality gates, post-launch monitoring, and continuous improvement • Work directly with the AI Quality lead, Igor Kolodkin • Collaborate closely with AI engineers, Product, and domain experts across the company • Work with Databricks, DeepEval, and Claude Code

🎯 Requirements

• Python and SQL — ability to build an analysis end-to-end • Solid foundation in statistics, including sampling, hypothesis testing, variance, and noisy metrics • Analytical mindset, starting from the business question rather than the tool • 3+ years in analyst or data scientist roles • At least 1 year in a product context • Experience in quality analytics for ML systems is a plus • Hands-on experience evaluating LLM applications (RAG, agents, tool use, judges) is a plus • Experience building LLM agents is a plus • AI-assisted coding is the default authoring environment • Claude Code is the main tool for SQL, Python, analyses, dashboards, and internal scripts • Already curious and fluent with AI coding, or genuinely excited to become fluent quickly

🏖️ Benefits

• Continuous personal and professional development resources and opportunities • Flexible traveling and remote or hybrid work across Europe • Stock Options Program available to every team member • Modern, friendly, and eco-conscious corporate culture • Constant support and care • One month in a corporate apartment in Cyprus through the Work & Swim Program