Product Data Scientist – AI Evaluation, Quality

🔥 0 minutes ago

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Finom

WebsiteLinkedInAll Job Openings

501 - 1000 employees

Founded 2019

💳 Fintech

💸 Finance

🤝 B2B

Fintech • Finance • B2B

Finom is a fintech company offering comprehensive financial services for SMEs, freelancers, and companies. The platform provides tools for invoicing, expense management, and easy online account opening with added benefits like cashback and international IBAN accounts. Finom ensures data security through partnerships with banks like SolarisBank and Treezor, making it a fast and reliable service for business financial management.

📋 Description

• Own and extend our offline eval suite across products — datasets (capability + regression), judges, metrics • Build and maintain online quality dashboards: resolution rate, CSAT, thumbs up/down, LLM-as-judge signals, error rate, latency • Close the production feedback loop: mine failure patterns from real traffic → turn them into regression cases → propose fixes to Product and domain experts • Harden methodology: judge stability, non-determinism handling • Translate numbers into decisions – weekly syncs, clear trade-offs, no dashboards for their own sake

🎯 Requirements

• Python and SQL — you can build an analysis end-to-end • Solid foundation in statistics — sampling, hypothesis testing, variance, understanding what a noisy metric is • Analytical mindset — you start from the business question, not from the tool • 3+ years in analyst / data scientist roles, at least one in a product context • Experience in quality analytics for ML systems — ranking, recommendations, classification, etc. • Hands-on experience evaluating LLM applications (RAG, agents, tool use, judges) • Experience building LLM agents — side projects, toy builds, personal experiments all count • AI-assisted coding is our default authoring environment • Fluent with AI coding or genuinely excited to become fluent fast • We care about what you ship and how clearly you think

🏖️ Benefits

• Make a genuine impact on the product • Join our upward trajectory • Continuous personal and professional development • Empowering you to make a genuine impact on our evolving product • Flexibility of traveling and working remotely or in a hybrid model across Europe • Become a stock options holder • Stock Options Program available to every team member • Constant support and care to ensure your Finom experience is successful and fulfilling • Work & Swim Program in Cyprus for work-life balance