Senior Software Engineer, AI Benchmarking

Job not on LinkedIn

🕒 July 27

🏄 California, Massachusetts – Remote

🏄 California – Remote 🍂 Massachusetts – Remote

💵 $180k - $245k / year

⏰ Full Time

🟠 Senior

🤖 AI Engineer

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

SecureBio

WebsiteLinkedInAll Job Openings

11 - 50 employees

🧬 Biotechnology

🔬 Science

🤝 Non-profit

Biotechnology • Science • Non-profit

SecureBio is a nonprofit organization that works to secure the world against catastrophic pandemics by developing technology, policy, and operational solutions to delay, detect, and defend against biological threats. Its in-house researchers and partners in academia, industry, and government focus on pathogen-agnostic early warning (the Nucleic Acid Observatory using deep metagenomic sequencing), assessing and mitigating AI-driven biotechnology risks, and creating tools like SecureDNA for DNA synthesis screening. SecureBio also explores physical interventions (e. g. , PPE and far-UVC) and policy proposals to improve biosafety, biosecurity, and societal resilience to engineered or natural pandemics.

📋 Description

• Perform pre-release and post-release assessments of frontier and open-source AI models using SecureBio's suite of biosecurity evaluations. • Build adapters for models, identify and troubleshoot unexpected agent behaviors. • Build, scale, maintain, and continuously improve SecureBio's suite of biosecurity evaluations. • Improve our evaluations to use state-of-the-art agent frameworks, jailbreaking methods, and elicitation strategies. • Develop and improve internal tools that enable our researchers to run capability assessments at scale. • Contribute to and maintain cloud infrastructure to run our evaluations and analyses. • Contribute analyses to our public dashboard of trends in AI biology capabilities.

🎯 Requirements

• A proven track record in software engineering, including 5+ years of experience in industry roles and/or open-source contributions. • Fluency in Python. • Proficiency with Docker and AWS (particularly ECS). • Experience building software tools that use or evaluate LLMs or AI agents. • Self-directed and comfortable working on small, fast-moving teams. • A commitment to good coding practices, including knowing when and how to use (or not use) coding agents. • Care about code quality, readability, and maintaining human oversight of human- and AI-written code. • Alignment with SecureBio's mission of preventing catastrophic pandemics.