Software Engineer, Agent Optimization – Evaluation

🔥 0 minutes ago

🇺🇸 United States – Remote

💵 $109.8k - $325k / year

⏰ Full Time

🟡 Mid-level

🟠 Senior

🧑‍💻 Full-stack Engineer

👻 Ghost score 0%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Reddit, Inc.

WebsiteLinkedInAll Job Openings

501 - 1000 employees

Founded 2005

💼 Consulting

📣 Marketing

📱 Media

Consulting • Marketing • Media

Reddit, Inc. is a social media platform that acts as a hub for thousands of communities, where users can engage in diverse conversations ranging from breaking news to niche interests. It enables users to post, comment, and vote on content, fostering a vibrant online community. Millions of people globally connect and share their passions on Reddit, creating a dynamic environment for authentic human interaction.

📋 Description

• Lead Reddit's agent optimization and evaluation platform • Design and build an agent optimization harness for evaluating, benchmarking, and improving AI agents across software engineering workflows • Develop Reddit-specific evaluation environments for measuring agent performance against meaningful engineering tasks • Build systems for offline replay, automated evaluation, synthetic task generation, regression detection, and human-in-the-loop feedback • Design model routing and orchestration strategies optimizing quality, latency, and cost • Define metrics and experimentation frameworks measuring developer productivity and engineering velocity • Build tooling to analyze agent behavior, investigate failures, and accelerate iteration • Partner with Platform Engineering, Infrastructure, and Developer Productivity teams • Shape Reddit's long-term strategy for a flexible, model-agnostic agent platform

🎯 Requirements

• 5-12+ years of engineering experience • Extensive experience building and shipping large-scale production software systems • Experience designing evaluation, experimentation, benchmarking, or measurement systems for AI, search, developer platforms, or other complex distributed systems • Strong understanding of modern LLMs, AI agents, and the engineering systems required to deploy them effectively • Experience building systems that optimize across quality, latency, reliability, and cost • Excellent software engineering fundamentals • Ability to operate independently in ambiguous environments and drive technical direction for highly strategic initiatives • Strong cross-functional collaboration skills and experience working across engineering, infrastructure, and platform organizations

🏖️ Benefits

• Comprehensive Healthcare Benefits and Income Replacement Programs • 401k with Employer Match • Global Benefit programs that fit your lifestyle, from workspace to professional development to caregiving support • Family Planning Support • Gender-Affirming Care • Mental Health & Coaching Benefits • Flexible Vacation & Paid Volunteer Time Off • Generous Paid Parental Leave • Equity in the form of restricted stock units • Medical, dental, and vision insurance • Flexible vacation and generous time off for vacation • Parental leave