Task Development Engineer

Job not on LinkedIn

πŸ•’ August 6

🌏 Anywhere in the World

πŸ’΅ $150 - $300 / hour

⏳ Contract/Temporary

🟠 Senior

πŸ‘·πŸ»β€β™€οΈ Engineer

Where this job comes from

Posted on METR's careers page β†—METR's careers page, not through a job board

First seen by Remote Rocketship August 6

Still open on METR's site. Checked 6 hours ago.

Apply Now
Logo of METR

METR

11 - 50 employees

Founded 2022

πŸ€– Artificial Intelligence

☁️ SaaS

Artificial Intelligence β€’ Research β€’ SaaS

METR is a research organization focused on evaluating and understanding the capabilities and risks of frontier AI systems. They develop and run tests to measure AI performance, particularly in autonomous capabilities and AI R&D. By collaborating with leading AI companies like Anthropic and OpenAI, METR aims to enhance third-party evaluations to manage the emerging risks of advanced AI technologies comprehensively.

πŸ“‹ Description

β€’ Develop difficult, novel tasks for AI models that remain challenging as model time horizons grow β€’ Perform quality assurance for existing tasks, verifying solvability and that models receive only the necessary information β€’ Baseline tasks within areas of expertise when helpful β€’ Score task completions from AIs or human baseliners β€’ Improve task development infrastructure and workflows β€’ Contribute to evaluations of frontier AI systems supporting METR’s Time Horizons methodology

🎯 Requirements

β€’ Several years of experience working on complex software engineering projects and codebases β€’ Experience building hard, ideally agent-based, AI evaluations β€’ Experience with evaluations such as RE-Bench, HCAST, SWE-bench Verified, Cybench, or GPQA β€’ Ideally experience using the Inspect framework β€’ High attention to detail, including spotting misspecifications and ambiguity β€’ Familiarity with METR infrastructure, including Hawk, is a plus β€’ Familiarity with the methodology behind METR’s Time Horizons work is a plus β€’ Availability for 20–40 hours per week β€’ Minimum of 1 hour of overlap with the Pacific Coast Time workday

πŸ–οΈ Benefits

β€’ Flexible schedule determined by you β€’ Remote work worldwide β€’ Flexible working hours of 20–40 hours per week β€’ Individuals who contribute >80 hours will be acknowledged in the final research output (if desired)

Apply Now

πŸ“Š Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Similar Jobs

πŸ•’ June 26

Vozy

51 - 200

πŸ€– Artificial Intelligence

☁️ SaaS

πŸ”’ Cybersecurity

Forward Deployed Engineer at Vozy deploying and optimizing AI voice agents for customer resolutions. Collaborate with clients and manage AI integrations in a dynamic environment.

🌏 Anywhere in the World

πŸ’° $1M Debt Financing on 2022-10

⏳ Contract/Temporary

🟑 Mid-level

🟠 Senior

πŸ‘·πŸ»β€β™€οΈ Engineer

πŸ—£οΈπŸ‡ͺπŸ‡Έ Spanish Required

Developed byΒ Lior Neu-ner. I'd love to hear your feedback β€” Get in touch via DM or support@remoterocketship.com
Follow Remote Rocketship on LinkedInFollow Remote Rocketship on InstagramFollow Remote Rocketship on FacebookFollow Remote Rocketship on TikTok