Senior Evaluation Algorithm Engineer – APAC

🔥 0 minutes ago

🌐 Australia, China, +1 more countries – Remote

🇦🇺 Australia – Remote 🇨🇳 China – Remote 🇹🇼 Taiwan – Remote

⏰ Full Time

🟠 Senior

👷🏻‍♀️ Engineer

👻 Ghost score 11%

What it means: the chance that no one is really hiring for this job.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Binance

WebsiteLinkedInAll Job Openings

1001 - 5000 employees

Founded 2017

₿ Crypto

💳 Fintech

💰 Initial Coin Offering on 2020-12

Crypto • Fintech

Binance is the world's leading cryptocurrency exchange, serving over 235 million registered users across more than 180 countries. The platform offers a wide array of services, including the trading of over 350 cryptocurrencies in Spot, Margin, and Futures markets. Users can also buy and sell crypto via Binance P2P, earn interest through Binance Earn, and engage in NFT trading on the Binance NFT marketplace. Binance provides low transaction fees and diverse payment options, making it a preferred choice for cryptocurrency enthusiasts worldwide.

📋 Description

• Design end-to-end LLM evaluation plans for dialogue, financial trading, and other business scenarios • Build evaluation metric systems and rubrics that produce quantifiable, reproducible, and explainable conclusions • Lead evaluation dataset design and construction, including evaluation dimensions, scenario coverage, annotation guidelines, and quality control • Analyze model capability boundaries and failure modes and produce actionable improvement recommendations • Collaborate with algorithm and product teams to drive model iteration • Automate and scale evaluation workflows • Build sustainable evaluation platforms and toolchains • Collaborate with algorithm, product, and data teams to translate objectives into evaluation standards and R&D directions

🎯 Requirements

• Master's degree or above in Computer Science, Artificial Intelligence, Mathematics, Statistics, or related fields • Solid algorithmic foundation and understanding of LLM principles, training, and fine-tuning processes • Hands-on LLM evaluation experience at a large tech company • Experience with commercial deployment evaluation, not purely academic or offline benchmarking • Familiarity with the full pipeline from evaluation data preparation and rubrics design to evaluation-driven R&D • Familiarity with human evaluation, model-based automatic evaluation / LLM-as-a-judge, and metric computation • Ability to define evaluation dimensions and write clear, actionable, and discriminative rubrics • Systematic control over evaluation data representativeness, annotation consistency, and result reliability • Proficient in Python • Experience in evaluation workflow automation, benchmark construction, or evaluation platform development • Ability to independently handle data processing, evaluation script writing, and result analysis • Strong business understanding and communication skills • Bonus: Experience evaluating dialogue systems, AI Agents, or financial/trading LLMs • Bonus: Experience building high-quality AI training/evaluation data or data annotation systems • Bonus: Familiarity with RLHF, reward models, or preference data-related work

🏖️ Benefits

• Shape the future of finance • Collaborate with world-class talent in a user-centric global organization with a flat structure • Tackle challenging projects in a fast-paced and innovative enviornment with autonomy • Thrive in a results-driven workplace with opportunities for career growth and continuous learning • Competitive salary and company benefits • Remote-first, work-from-home arrangement (may vary depending on the nature of the role and team)

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume