Senior Software Engineer, AI Evaluation Infrastructure

Job not on LinkedIn

🔥 12 hours ago

🇺🇸 United States – Remote

⏰ Full Time

🟠 Senior

🤖 AI Engineer

🦅 H1B Visa Sponsor

This company has sponsored H1B visas in the past.

👻 Ghost score 10%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Turing

WebsiteLinkedInAll Job Openings

1001 - 5000 employees

Founded 2018

💼 Consulting

📦 Logistics

📣 Marketing

Consulting • Logistics • Marketing

Turing is a leading company in the field of artificial intelligence, specializing in the advancement of AGI (Artificial General Intelligence) and the evaluation and training of large language models (LLMs). Turing offers a comprehensive range of AI solutions, including model performance assessments, multimodal capabilities integration, and custom engineering for enterprises looking to optimize their AI operations. The company bridges cutting-edge AI research with practical applications, empowering organizations to leverage advanced AI for various real-world challenges.

📋 Description

• Design and ship end-to-end features across the data model, API, background workers, and UI • Build and improve agent and model evaluations in sandboxed environments • Score evaluation results and turn thousands of runs into trustworthy results • Improve benchmark task quality checks using verifiers, rubric-based grading, and LLM-as-a-judge grading • Integrate new models, agents, and benchmarks through generalizable designs • Improve system performance and reliability through efficient queries, safe migrations, and robust concurrency • Review teammates’ pull requests against engineering guidelines • Deliver tested, maintainable changes and independently own system areas

🎯 Requirements

• Typically 5+ years building and operating production backend systems • Ownership of at least one service from design through on-call • Strong Python • Experience with Django or a similar framework, PostgreSQL, and a task queue such as Celery • Hands-on experience in LLM evaluation, including benchmark task construction, agent trial scoring, and result aggregation/comparison • Judgment about task quality, including ambiguous instructions, weak or gameable verifiers, leaked answers, and invalid checks • Ability to make sound tradeoffs and choose the smallest effective design • Ownership of migrations, tests, and edge cases • Ability to quickly adopt team conventions for naming, structure, tests, and error handling • Comfort with TypeScript and React frontend work • Clear technical writing in pull request descriptions and design notes • Nice to have: Docker, Kubernetes, Modal, Daytona, LLM-as-a-judge, rubric-based grading, Claude Code, Codex, OpenHands, MCP, distributed job scheduling, concurrency limits, database locking at scale, and open-source evaluation or infrastructure contributions

🏖️ Benefits

• Work at the frontier of AI, helping leading AI labs improve advanced models by building expert datasets, RL environments, and benchmarks • Contribute to leading-edge AI research and showcase work at conferences such as ICLR, ICML, and NeurIPS • Bring frontier AI innovation to enterprise challenges • Collaborate with and learn from colleagues with deep AI experience from Google, Meta, Amazon, and other leading technology companies • Startup-speed environment with speed, ownership, and impact • Equal opportunity, diverse, inclusive, and authentic workplace