Senior Manager, Data Quality – Evaluation

Job not on LinkedIn

🔥 0 minutes ago

Apply Now

Find Similar Remote Jobs

Receive Emails For Remote Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Argos Multilingual

WebsiteLinkedInAll Job Openings

201 - 500 employees

🤝 B2B

☁️ SaaS

🤖 Artificial Intelligence

B2B • SaaS • Artificial Intelligence

Argos Multilingual is a global language-services provider that delivers translation, localization, and multilingual content solutions combining human expertise with AI-first technology. The company offers translation and localization, multimedia services, language quality evaluation and ownership programs, content and linguistic asset management, and AI/LLM data solutions (data collection, annotation, evaluation, RLHF, and custom machine translation) through its MosAIQ suite of enterprise tools. Argos serves B2B customers across regulated and technical sectors—life sciences, medical/healthcare, technology, finance, manufacturing, and retail—focusing on accuracy, compliance, and scalable AI-enabled workflows.

📋 Description

• Build quality systems for AI data programs • Design and manage quality frameworks for AI data and evaluation programs. • Translate customer requirements into clear quality standards, rubrics, acceptance criteria, review processes, and KPIs. • Build quality workflows that are practical, scalable, and trusted by customers as programs move from pilot to production. • Identify quality risks early and work with delivery teams to resolve issues before they impact timelines, customer confidence, or program outcomes. • Create repeatable quality processes across calibration, QA sampling, adjudication, reviewer performance tracking, and customer reporting. • Lead evaluation, calibration, and QA processes • Support quality operations across multilingual evaluation, speech/audio QA, transcription, data annotation, human preference evaluation, expert review, model response evaluation, coding evaluation, tool-use evaluation, and agent workflow evaluation. • Create and improve rubrics, task instructions, reviewer guides, calibration exercises, golden datasets, and quality reporting templates. • Lead calibration sessions with reviewers, annotators, quality specialists, delivery teams, and customer stakeholders. • Define quality thresholds, error taxonomies, escalation rules, and corrective action plans. • Monitor reviewer agreement, disagreement trends, error rates, contributor performance, and root causes of quality variance. • Turn QA findings into practical improvements to instructions, training, tooling, staffing, and delivery workflows. • Partner with customers and internal teams • Act as a quality lead for strategic customer programs when needed. • Support customer-facing quality readouts, pilot retrospectives, business reviews, escalations, and scale-up discussions. • Provide clear, data-backed reporting that explains quality performance, risks, corrective actions, and next steps. • Partner with Program Management, Supply Chain, Solutions, Sales, and Operations to ensure programs are set up for quality success from the start. • Work with Supply Chain to define reviewer profiles, evaluator requirements, language requirements, domain expertise, onboarding needs, and performance expectations. • Help determine when programs require expert reviewers, QA leads, language leads, technical reviewers, or specialized evaluation talent. • Build and develop the quality function • Build reusable quality assets such as calibration packs, QA reports, rubric libraries, error taxonomies, scorecards, and sample evaluation frameworks. • Identify repeatable patterns across programs and turn them into standardized approaches that help the business scale. • Improve visibility into quality performance across programs, reviewers, contributors, and workflows. • Manage, coach, and support Quality Managers, Quality Leads, Quality Specialists, reviewers, or QA contributors assigned to Data Services programs. • Coach team members on quality judgment, customer communication, escalation handling, reporting, and root-cause analysis. • Identify hiring, training, and coverage needs as the Data Services business grows. • Create a culture of quality ownership, accountability, and continuous improvement. • Anticipate and communicate the needs identified for the team under your responsibility. • Train, and support team members, advocate for upskilling and promote career growth. • Be responsible for offering help to team members during increased workload periods, helping to avoid risks to Client deliveries due to their time demands (this includes finding cover for sickness and absence).

🎯 Requirements

• 5+ years of experience in quality operations, data operations, AI data services, localization quality, annotation quality, evaluation operations, trust and safety quality, or a related field. • Experience managing quality programs for complex customer accounts or high-volume operational delivery. • Strong understanding of QA methodologies, calibration, sampling, adjudication, error analysis, and performance reporting. • Experience working cross-functionally with delivery, operations, supply chain, sales, and customer-facing teams. • Strong analytical skills and the ability to turn quality data into clear operational improvements. • Excellent written and verbal communication skills, including the ability to communicate quality issues clearly to customers and senior stakeholders. • Comfort operating in fast-moving, ambiguous environments where processes are still being built. • Strong people leadership skills with experience coaching quality specialists, reviewers, annotators, or operational teams.

🏖️ Benefits

• Professional development opportunities