Senior Software Engineer – AI Evaluation & Benchmarks, Python

Job not on LinkedIn

🕒 May 14

🐊 Florida – Remote

🐊 Florida – Remote

💵 $80 - $100 / hour

⏳ Contract/Temporary

🟠 Senior

🔙 Backend Engineer

Apply Now

Find Similar Remote Jobs

Receive Emails For Remote Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

G2i Inc.

WebsiteLinkedInAll Job Openings

11 - 50 employees

🎯 Recruiter

🏢 Enterprise

☁️ SaaS

Recruitment • Enterprise • SaaS

G2i Inc. is a video-based platform specializing in the hiring of contract or full-time engineers. With a vast pool of over 8,000 engineers, G2i offers quality matches quickly by customizing its sourcing process based on client needs. They provide a 7-day free trial to engage potential hires before making a commitment. The platform emphasizes speed and quality, utilizing video assessments and AI training to ensure engineers meet specific technical standards. G2i handles compliance, international payments, and background checks, providing a streamlined hiring process for companies looking to hire in the US, Canada, Latin America, or Europe. By focusing on technical roles such as JavaScript, Python, iOS, Android, and AI engineers, G2i ensures companies can hire top talent efficiently.

📋 Description

• Design and build the coding benchmarks and evaluation pipelines used to test frontier AI models on real software engineering work: • Design coding benchmarks that evaluate frontier models on real-world programming tasks — reasoning, debugging, and production-quality code • Build and maintain scalable data pipelines for evaluation workflows • Analyze model-generated code for correctness, reliability, and edge-case failures • Construct structured evaluation scenarios across large repos and multi-language environments • Provide detailed technical feedback on model performance and failure patterns • Contribute to evaluation frameworks that set the bar for how coding ability is measured • End result: benchmarks that meaningfully separate what frontier models can and can't do — and shape how the next generation is trained and improved. • AI coding evaluation in one line: Design task → build harness → run model → analyze failures → feed findings back into the benchmark → evaluations that actually distinguish strong models from weak ones.

🎯 Requirements

• 4+ years of professional software engineering experience (non-negotiable) • Expert Python — clean, performant, well-tested code • Hands-on experience working in large, complex codebases • Proven experience designing and implementing LLM coding benchmarks and evaluation data pipelines • Strong command of Git and modern development workflows • Track record at a high-growth tech company or top-tier software organization • Strong written English communication. • Identity verification: Applicants must verify identity and have valid documentation to work as an independent contractor.

🏖️ Benefits

• Identity verification required for independent contractors in residence country • Weekly payments via PayPal or Stripe