Senior QA Engineer – AI

🔥 12 hours ago

🇮🇳 India – Remote

⏰ Full Time

🟠 Senior

🔧 QA Engineer (Quality Assurance)

👻 Ghost score 10%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Delta Exchange

WebsiteLinkedInAll Job Openings

51 - 200 employees

Founded 2018

₿ Crypto

💳 Fintech

🏪 Marketplace

💰 $5M Initial Coin Offering - Delta Exchange on 2021-03

Crypto • Fintech • Marketplace

Delta Exchange is a crypto derivatives exchange offering futures and options trading for major cryptocurrencies such as Bitcoin and Ether. It provides 24/7 markets with efficient margining, INR-denominated margin and P/L, and INR deposit/withdrawal and settlement, allowing users in India to trade crypto derivatives without owning the underlying assets. The platform is registered with the FIU (Government of India) and presents features like an algo marketplace, token listings including RWA tokens, and pro trading tools optimized for mobile and web.

📋 Description

• Build and maintain golden datasets for AI products, sourced from production traces or generated and verified against a documented source of truth • Design layered scoring for AI outputs using deterministic rules and model-graded evaluation • Manage evaluation runs end to end, including scheduling, execution, baseline comparison, and failure triage • Own and operate the release gate for prompt or model changes • Identify failure modes including hallucination, incorrect tool selection, loss of conversational context, and PII exposure • Investigate production traces to distinguish retrieval, generation, and tool failures • Convert findings into actionable engineering evidence and permanent regression coverage • Define and report quality metrics for AI surfaces and drive improvement

🎯 Requirements

• At least 1 year owning quality for AI products in a lead or primary-owner capacity • Direct experience with prompts, tool and function schemas, and model behaviour • Hands-on experience building or operating an evaluation harness for an AI system, using frameworks such as DeepEval, RAGAS or Promptfoo, or an in-house solution • 4-6 years of QA / SDET experience • Working proficiency in Python • Ability to analyse traces and tool calls using observability tooling such as Opik, Langfuse or LangSmith, or an equivalent • Strong API testing experience • Sufficient engineering ability to build tooling and automation • Clear written communication • Experience with trading or exchange platforms, including futures and options, crypto derivatives, margin, or settlement flows is a bonus

🏖️ Benefits

• Competitive compensation • Significant growth opportunities • Leadership exposure • Work directly with founders and senior leadership on high-impact strategic initiatives • Opportunity to shape the regulatory landscape for digital assets and Web3 in India • Mission-driven, fast-growing organisation at the forefront of financial innovation