Benchmark & Evaluation Engineer

Job not on LinkedIn

🕒 Yesterday

🇵🇭 Philippines – Remote

⏰ Full Time

🟡 Mid-level

🟠 Senior

👷🏻‍♀️ Engineer

👻 Ghost score 13%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Expedock

WebsiteLinkedInAll Job Openings

51 - 200 employees

📦 Logistics

🤖 Artificial Intelligence

🚗 Transport

Logistics • Artificial Intelligence • Transport

Expedock is the AI-powered automation service behind some of the leading players in the 7 trillion USD global supply chain. The company is on a mission to build a powerful data infrastructure that drives unprecedented efficiency and profitability across the industry. Expedock focuses on automating the manual processing and inputting of data for freight forwarding and logistics, achieving 99. 99% accuracy. With a team of experts in artificial intelligence and logistics, Expedock aims to transform the future of supply chain and logistics.

📋 Description

• Design and stand up multi-system enterprise environments, including warehouses, lakes, APIs, and databases, at scale • Build environments across domains such as healthcare, finance, and product analytics • Generate calibrated, large-scale benchmark datasets with realistic noise, seasonality, drift, and synthetic PII • Test query scalability and reasoning using benchmark datasets • Write and verify benchmark tasks, golden SQL, and rubric-scored reasoning keys • Eliminate data leakage and ambiguity from benchmark tasks • Extend grading systems using deterministic checks and LLM judges • Maintain regression-tracked leaderboards • Ensure evaluation scoring integrity

🎯 Requirements

• Expert proficiency in production-grade SQL, including CTEs, window functions, query plans, and cardinality • Expertise in data modeling, including star/snowflake schemas, SCDs, and referential integrity • Strong Python skills with pandas, NumPy, and Polars • Hands-on experience standing up and loading data into at least one major warehouse: Snowflake, BigQuery, Redshift, or Postgres • Proven experience designing evaluation sets or benchmark tasks • Experience with golden-answer/golden-SQL verification, rubric/LLM-judge calibration, avoiding data leakage, and tracking regressions • Ability to construct realistic multi-system setups including warehouses, data lakes, operational databases, and APIs • Experience with large-scale, semi-structured, or dirty synthetic data • Domain knowledge in Healthcare/Health Insurance, Finance/FP&A, Product Analytics, or Supply Chain is nice to have • Experience with LLM evaluation frameworks or streaming/data-lake stacks is nice to have

🏖️ Benefits

• Remote work in the Philippines • US Shift schedule