Senior AI Backend Engineer – Agent Evaluation, Quality

🕒 July 19

🇸🇦 Saudi Arabia – Remote

⏰ Full Time

🟠 Senior

🔙 Backend Engineer

👻 Ghost score 10%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Salla E-Commerce Platform

WebsiteLinkedInAll Job Openings

51 - 200 employees

🛍️ eCommerce

☁️ SaaS

eCommerce • SaaS

Salla E-Commerce Platform is an innovative platform designed to simplify the process of starting and managing online businesses. It specializes in e-commerce solutions, enabling users to create and operate their online shops effortlessly. With a focus on providing accessible tools and services for e-commerce, Salla is shaping the future of online commerce in a user-friendly manner.

📋 Description

• Own the evaluation stack. Design and build LLM-as-judge systems, calibrate them against human labels, and make agent quality measurable per-agent and per-failure-mode. • Make the release gate real. Build per-PR eval harnesses and regression detection wired into CI, so quality is enforced automatically, not by manual passes. • Build user simulators to generate test coverage and adversarial cases before real users hit them. • Turn production signal into improvement - pipe real failures back into evaluation sets so the system compounds over time. • Partner with product to turn "what good looks like" into concrete, measurable criteria. • Grow into agent development - contribute to building and hardening the agents themselves, starting with the components you know most deeply from evaluating them.

🎯 Requirements

• Strong software engineering fundamentals. Production Python or Typescript (or similar), clean API and system design, testing, CI/CD. You write code others build on - evaluation infrastructure is real engineering. • Hands-on LLM/agent experience. You've built with LLMs - agents, RAG, tool/function calling, orchestration frameworks (LangGraph, LangChain, or equivalent) - and understand how they behave and break. • A measurement mindset. You reason about metrics, calibration, and experiments; you want to quantify whether something works, not just ship it. • Production experience. You've run LLM systems in production and dealt with reliability, latency, cost, and observability. • 5+ years software engineering, with recent hands-on LLM/agent work. • Nice to have • Direct experience evaluating LLM/agent systems - offline/online eval, LLM-as-judge, systematic regression testing. • Observability tooling (Arize, LangSmith, or similar). • Arabic language / NLP experience. • E-commerce or merchant-facing product experience.