Expert Software Engineer – AI Evaluation

Job not on LinkedIn

🔥 2 minutes ago

🇮🇳 India – Remote

💵 $50 - $100 / hour

⏳ Contract/Temporary

🟢 Junior

🟡 Mid-level

🤖 AI Engineer

👻 Ghost score 0%

What it means: the chance that no one is really hiring for this job.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Patronus AI

WebsiteLinkedInAll Job Openings

51 - 200 employees

Founded 2023

🤖 Artificial Intelligence

🔬 Science

🔥 Funding within the last year

💰 $50M Series B on 2026-06

Artificial Intelligence • Science

Patronus AI is a behavioral research lab for artificial intelligence that studies where models and agents fail and how to improve their judgment. They build realistic, domain-specific simulated environments and world models where agents can practice decision-making under fragmented information, conflicting requests, and unstated requirements; analyze agent mistakes; and create training data, benchmarks, and tests (e. g. , TRAIL, MEMTRACK) to improve AI reliability and alignment. The company emphasizes fundamental research, publishes on arXiv, and provides resources such as RL environments, testing frameworks, and domain guides. Their site references enterprise-relevant compliance (SOC 2, HIPAA, TISAX).

📋 Description

• Design realistic, multi-step coding and technical tasks that test AI models' reasoning, coding, and instruction-following • Build tasks using codebases, technical specifications, architecture documents, and datasets • Evaluate AI-generated code and technical content for correctness, efficiency, clarity, and real-world engineering alignment • Grade model outputs against rubrics and ground-truth answers and write structured feedback • Improve AI understanding of programming, system design, data workflows, and engineering best practices • Work independently and asynchronously, meet deadlines, and collaborate with the Patronus team • Test and refine content to reflect real-world complexity • Review other contributors' work for accuracy, clarity, and practical relevance • Incorporate feedback promptly and flag gaps

🎯 Requirements

• 2+ years of professional experience in software engineering; internships excluded • Experience with system design, data pipelines, ML or LLM-based applications • Prior work in code review, technical writing, AI training or model evaluation • Strong coding skills in at least one major language, such as Python, Java, C++, JavaScript, TypeScript or Go • Experience building, reviewing and debugging production-grade code • Ability to follow detailed technical guidelines, work independently, respond to feedback, and meet deadlines • Excellent written communication and close attention to detail • A bachelor's degree in computer science, engineering or a related field, or equivalent experience • All tasks, project files and solutions must be human-authored; LLM-generated code, content or text is not permitted • Tasks and solutions must be original and not copied from published or easily searchable sources • Must not include code, assets or project files from a current or former employer, client or studio, or anything covered by an NDA • Work must use fictional studios, projects and scenarios, with all code and content owned by the contributor

🏖️ Benefits

• Flexible commitment, typically 15–20 hours per week • Can be done alongside a full-time job • Task-based contract with some tasks paid at a flat rate • Flexible, project/task-based work

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume