QA Auditor, Sampling, Quality Gates, Release Readiness

🕒 February 27

Apply Now

Find Similar Remote Jobs

Receive Emails For Remote Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Codvo.ai

WebsiteLinkedInAll Job Openings

51 - 200 employees

Founded 2019

🔒 Cybersecurity

☁️ SaaS

AI • Cybersecurity • SaaS

Codvo. ai is a technology company that specializes in delivering strategic enterprise solutions through advanced AI-driven innovation. They focus on transforming enterprise data into measurable value by helping businesses accelerate growth with custom AI implementations tailored to meet the specific challenges of various industries. Their extensive service offerings include AI/ML automation, application development, data analytics, cybersecurity, and digital transformation, ensuring that organizations can thrive in a rapidly evolving digital landscape.

📋 Description

• Protect dataset quality at scale by running sampling-based audits, enforcing quality gates, and ensuring every dataset release is audit-ready and meets acceptance thresholds. • Execute sampling plans (stratified by label type, difficulty, site/surgeon/device, model confidence) for ongoing QA and release audits. • Audit annotations against guidelines + ontology; score defects using standardized rubrics (severity, type, root cause). • Run and validate quality gates prior to release: completeness/coverage checks schema/constraint checks (illegal combinations, temporal consistency, boundary rules) defect density and rework thresholds. • Generate rework tickets with precise instructions; track closure and verify fixes. • Partner with Annotation Lead to drive calibration actions (training refreshers, guideline clarifications, examples library). • Maintain QA dashboards: defect rate, rework %, gate pass rate, audit cycle time, top error patterns by annotator/team/label type. • Support release readiness: ensure required artifacts are complete for the Evidence Pack (QA results, sampling logs, sign-offs). • Escalate systematic issues to Adjudication/Clinical Review (definition gaps, ambiguous classes, recurring edge cases).

🎯 Requirements

• 3–7+ years in data QA / annotation QA / operations quality; experience auditing labeled datasets (video preferred). • Strong command of sampling-based QA: stratified sampling, acceptance thresholds, and audit scoring. • High attention to detail; comfortable reading and enforcing detailed SOPs/guidelines. • Strong documentation skills; can write clear defect descriptions and rework instructions. • Comfortable with dashboards/spreadsheets; basic metric literacy (defect density, rework, throughput).