Senior Machine Learning Engineer – Evaluations, Design Generation

2 hours ago

Apply Now

Receive Emails with Similar Jobs

Canva

WebsiteLinkedInAll Job Openings

SaaS • Media • Education

Canva is a versatile online design platform that empowers users to create a wide range of professional designs with ease. From social media posts and presentations to business cards and posters, Canva provides thousands of templates and design tools to help users bring their creative ideas to life. The platform also offers a suite of AI-powered features to enhance creativity and productivity, including tools like Magic Write for copy generation and Magic Edit for photo transformations. Canva caters to individuals, teams, and enterprises, making it an ideal solution for collaborative design and workflow management. It is also committed to sustainability and social impact, offering free educational and nonprofit access to its premium features.

1001 - 5000 employees

Founded 2013

☁️ SaaS

📱 Media

📚 Education

💰 $200M Venture Round on 2021-09

📋 Description

• Understand and optimize existing evaluation systems including LLM-as-Judge frameworks, visual quality models , and multi-dimensional scoring approaches - analyzing their strengths, limitations, and trade-offs to identify gaps in coverage and opportunities for improvement across brand adherence, visual appeal, layout quality, and functional correctness • Design and implement automated evaluation pipelines that score generated designs at scale, balancing accuracy with computational cost • Define evaluation strategies for different scenarios: pre-deployment validation, continuous monitoring, A/B experiment analysis, and model comparison • Curate high-quality evaluation datasets and benchmark suites that represent diverse use cases, edge cases, and quality dimensions • Integrate evaluation systems into continuous deployment pipelines, creating automated quality gates that catch regressions before production • Reduce evaluation cycle time to enable teams to iterate faster on model improvements and launch experiments earlier • Partner with research teams to understand evaluation needs for new model architectures and capabilities • Define the evaluation ecosystem strategy: how different evaluation tools and methods compose together for Design Generation • Guide teams on evaluation best practices, appropriate methodologies for their use cases, and interpretation of results

🎯 Requirements

• Strong ML engineering fundamentals with experience building and maintaining production ML systems at scale • Proven ability to build robust, scalable infrastructure (not just models) - you're a platform engineer who speaks ML • Deep understanding of distributed systems, observability patterns, and monitoring best practices • Python proficiency with production-quality coding standards, code reviews, and testing practices • Experience with data pipelines, time-series data, and statistical analysis for detecting anomalies • SQL fluency for querying and analyzing large datasets across data warehouse and analytics systems • Track record of building self-service platforms or developer tooling that gets adoption • Excellent collaboration skills - this role requires working across teams to understand needs and deliver solutions • Experience with evaluation of Gen AI systems at scale (even better if that’s evaluation of systems with creative outputs!)

🏖️ Benefits

• Equity packages - we want our success to be yours too • Inclusive parental leave policy that supports all parents & carers • An annual Vibe & Thrive allowance to support your wellbeing, social connection, office setup & more • Flexible leave options that empower you to be a force for good, take time to recharge and supports you personally