Applied AI Software Engineer

🕒 June 19, 2025

Apply Now

Find Similar Remote Jobs

Receive Emails For Remote Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Canvas Medical

WebsiteLinkedInAll Job Openings

51 - 200 employees

Founded 2016

⚕️ Healthcare Insurance

☁️ SaaS

🤖 Artificial Intelligence

💰 $24M Series B on 2022-07

Healthcare Insurance • SaaS • Artificial Intelligence

Canvas Medical is a company that offers a suite of customized electronic medical records (EMR) solutions tailored to specific patient populations, operational settings, and payment models. Their products include specialized EMRs for cardiometabolic health, sleep medicine, urgent care telehealth, comprehensive diabetes care, and more. Canvas Medical provides tools for clinicians and developers, including a developer sandbox with support, prebuilt integrations, and a Software Development Kit (SDK) for building AI-powered protocols for healthcare. They also offer plans for clinics, builders, and enterprises, focusing on providing seamless workflows and integrated platforms for medical practice. Their systems are designed for quick mastery by clinicians and include features for billing, scheduling, and clinical record management with integrations facilitated by the FHIR API. Canvas Medical is committed to accelerating the practice of medicine through open core development and community collaboration.

📋 Description

• We’re hiring an Applied AI Software Engineer to lead evaluations for agents in development and the post-deployment fleet of agents operating in Canvas to automate work for our customers. • You will help develop agents in Canvas using state of the art foundation model inference and fine-tuning APIs along with our server-side SDK. • The server-side SDK provides extensive tools and virtually all the context necessary for excellent agent performance. • You’ll be responsible for designing and running rigorous evaluation experiments that measure performance, safety, and reliability across a wide variety of clinical, operational, and financial use cases. • This role is ideal for someone with deep experience evaluating LLM-based agents at scale. • You’ll create high-fidelity unit evals and end-to-end evaluations, define expert-determined ground truth outcomes, and manage iterations across model variants, prompts, tool use, and context window configurations. • Your work will directly inform model selection, fine-tuning, and go/no-go decisions for AI features used in production settings. • You’ll collaborate with product, ML engineering, and clinical informatics teams to ensure that Canvas's AI agents are not only capable, but trustworthy and robust under real-world healthcare constraints. • You will also work with technical product marketers and developer advocates to help our broader developer community and the broader market understand the uniquely differentiated value of agents in Canvas.

🎯 Requirements

• 5+ years of experience in applied machine learning or AI engineering, with a focus on evaluation and benchmarking. • Proficiency with foundation model APIs and experience orchestrating complex agent behaviors via prompts or tools. • Experience designing and running high-throughput evaluation pipelines, ideally including human-in-the-loop or expert-labeled benchmarks. • Superlative Python engineering skills and familiarity with experiment management tools and data engineering toolsets in general including, yes, SQL and database management. • Familiarity with clinical or healthcare data is a strong plus. • Experience with reinforcement fine-tuning, model monitoring, or RLHF is a plus. • Research shows that women and other minority groups might avoid applying if they don’t meet 100% of the qualifications. We encourage you to apply even if you don’t meet everything listed in the job posting.

🏖️ Benefits

• Competitive Salary & Equity Package • Health Insurance • Home Office Stipend • 401k • Paid Maternity/Paternity Leave (12 weeks) • Flexible/unlimited PTO