Product Data Scientist – AI Evaluation, Quality

🕒 July 3

🇷🇸 Serbia – Remote

⏰ Full Time

🟡 Mid-level

🟠 Senior

📊 Data Scientist

👻 Ghost score 29%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Finom

WebsiteLinkedInAll Job Openings

501 - 1000 employees

Founded 2019

💳 Fintech

💸 Finance

🤝 B2B

Fintech • Finance • B2B

Finom is a fintech company offering comprehensive financial services for SMEs, freelancers, and companies. The platform provides tools for invoicing, expense management, and easy online account opening with added benefits like cashback and international IBAN accounts. Finom ensures data security through partnerships with banks like SolarisBank and Treezor, making it a fast and reliable service for business financial management.

📋 Description

• Join the AI Team driving Finom’s AI products and technology • Own and extend the offline evaluation suite across products, including capability and regression datasets, judges, and metrics • Build and maintain online quality dashboards covering resolution rate, CSAT, thumbs up/down, LLM-as-judge signals, error rate, and latency • Close the production feedback loop by mining failure patterns from real traffic and turning them into regression cases • Propose fixes to Product and domain experts • Harden evaluation methodology, including judge stability and non-determinism handling • Translate metrics into decisions through weekly syncs and clear trade-offs • Own the evaluation loop across AI products, including pre-launch quality gates, post-launch monitoring, and continuous improvement • Collaborate closely with AI engineers, Product, and domain experts • Work directly with the AI Quality lead, Igor Kolodkin

🎯 Requirements

• Python and SQL — ability to build an analysis end-to-end • Solid foundation in statistics, including sampling, hypothesis testing, variance, and understanding noisy metrics • Analytical mindset focused on business questions • 3+ years in analyst or data scientist roles • At least 1 year in a product context • Experience in quality analytics for ML systems, such as ranking, recommendations, or classification (nice-to-have) • Hands-on experience evaluating LLM applications, including RAG, agents, tool use, and judges (nice-to-have) • Experience building LLM agents (nice-to-have) • Already curious and fluent with AI-assisted coding, or genuinely excited to become fluent quickly • Familiarity with Claude Code as the main authoring tool for SQL, Python, analyses, dashboards, and internal scripts

🏖️ Benefits

• Continuous personal and professional development resources and opportunities • Flexibility to travel and work remotely or in a hybrid model across Europe • Stock Options Program available to every team member • Constant support and care in a modern, friendly, eco-conscious corporate culture • One month in a corporate apartment in Cyprus through the Work & Swim Program