Qualitative Evaluation Engineer

🔥 47 minutes ago

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Luma AI

WebsiteLinkedInAll Job Openings

2 - 10 employees

🤖 Artificial Intelligence

☁️ SaaS

📱 Media

💰 $90M Series B on 2025-01

Artificial Intelligence • SaaS • Media

Luma AI is a Palo Alto–based company building multimodal generative AI models and creative workflow tools focused on video, image, audio, and text production. Luma offers models and products such as Ray (video generation), UNI-1/Uni-1. 1 (brand intelligence models), and an API/enterprise offerings that allow creative teams to generate, edit, and direct cinematic-quality assets with brand continuity. They emphasize frontier research (Open Physical AI Lab), in-house model development, and products for creative teams, agencies, and production services. Luma also provides APIs, enterprise plans, a creative partner program, and learning resources.

📋 Description

• Evaluate generative model performance across diverse tasks, prompts, and modalities, and surface the failure modes, regressions, and edge cases that hurt product quality • Build and maintain qualitative evaluation frameworks that are scalable and reusable • Translate high-level product goals into concrete evaluative criteria • Lead qualitative studies, side-by-side comparisons, and human-in-the-loop evaluations • Turn nuanced judgments into clear feedback that informs fine-tuning, dataset curation, and product UX • Work closely with technical artists and engineers to keep evaluations aligned with model capabilities and real use cases

🎯 Requirements

• 5+ years in product evaluation, UX research, model testing, or similar structured qualitative assessment • Master's or higher in Cognitive Science, HCI, Design Research, Psychology, Media Studies, or a related field • Systems thinking: you can define abstract qualities like believability or scene coherence in clear evaluative terms • Comfort working across engineers, researchers, and creatives

🏖️ Benefits

• Luma's mission is to build unified general intelligence that can generate, understand, and operate in the physical world • Excellent written communication and the ability to synthesize nuanced judgment into actionable insight • Deep familiarity with creative workflows for generative models (animation, filmmaking, digital art, VFX)