Python Engineer, AI Coding Agent Evaluator

Job not on LinkedIn

🔥 14 hours ago

🇺🇸 United States – Remote

💵 $100 - $200 / hour

⏳ Contract/Temporary

🟡 Mid-level

🟠 Senior

🔙 Backend Engineer

👻 Ghost score 7%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

G2i Inc.

WebsiteLinkedInAll Job Openings

11 - 50 employees

🎯 Recruiter

🏢 Enterprise

☁️ SaaS

Recruitment • Enterprise • SaaS

G2i Inc. is a video-based platform specializing in the hiring of contract or full-time engineers. With a vast pool of over 8,000 engineers, G2i offers quality matches quickly by customizing its sourcing process based on client needs. They provide a 7-day free trial to engage potential hires before making a commitment. The platform emphasizes speed and quality, utilizing video assessments and AI training to ensure engineers meet specific technical standards. G2i handles compliance, international payments, and background checks, providing a streamlined hiring process for companies looking to hire in the US, Canada, Latin America, or Europe. By focusing on technical roles such as JavaScript, Python, iOS, Android, and AI engineers, G2i ensures companies can hire top talent efficiently.

📋 Description

• Evaluate AI-generated coding interactions end-to-end • Assess whether responses make sense and whether preambles and reasoning are useful • Judge whether outputs demonstrate strong engineering judgment • Evaluate whether interactions feel appropriate to experienced developers • Assess usefulness, high-level correctness, and alignment with strong engineering thinking • Evaluate explanations and reasoning, not only code • Distinguish between different levels of response quality • Provide clear, opinionated feedback on what worked, what failed, and what felt misleading • Help define what excellent interactions look like for tools such as Cursor • Make subjective but rigorous judgments about AI coding-agent behavior

🎯 Requirements

• Highly experienced software engineer at Senior+ level • Staff / Principal-level engineer or equivalent experience • Strong background in TypeScript / JavaScript or Python • Hands-on experience using OpenAI Codex, Claude Code, and Cursor • Deep familiarity with modern AI-assisted development workflows • Able to evaluate code without fully executing or deeply reviewing every line • Comfortable giving direct, opinionated feedback • High bar for engineering quality • Take-home evaluation exercise • One behavioral interview • Simple background check for the project • Must select preferred programming languages from the available list

🏖️ Benefits

• $100–$200/hour rate • Approximately 10–20 hours/week • Possible extension beyond early May