Senior QA Engineer

🕒 October 6

🇺🇸 United States – Remote

⏰ Full Time

🟠 Senior

🔧 QA Engineer (Quality Assurance)

🦅 H1B Visa Sponsor

This company has sponsored H1B visas in the past.

👻 Ghost score 10%

What it means: the chance that no one is really hiring for this job.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Vi

WebsiteLinkedInAll Job Openings

51 - 200 employees

🏥 Healthcare

💼 Consulting

📦 Logistics

Healthcare • Consulting • Logistics

Vi is a company that leverages artificial intelligence to help health organizations improve member health outcomes and financial returns. The company provides an AI SaaS platform that offers solutions for efficient acquisition, personalized engagement, and predictive retention throughout the member lifecycle. Vi focuses on maximizing acquisition, enrollment, engagement, retention, upsells, and health outcomes with superior ROI. Their platform is designed to lower the cost per acquisition, enhance member engagement and retention, and transform operations with their generative AI technology. Vi works with leading global health enterprises to drive results at scale while ensuring privacy and data security standards are met.

📋 Description

• Own quality for voice agents that call real patients, providers, and caregivers • Build test and evaluation systems for engineering development • Define release criteria and determine what must pass before and after client launches • Analyze call recordings, transcripts, latency traces, and logs to isolate failures across speech-to-text, models, text-to-speech, telephony, and integrations • Create reproducible defect tickets from failed calls • Develop golden test sets, simulated calls, judged transcripts, classification and extraction accuracy tests, and structured-output validation • Set tolerances and maintain trustworthy regression coverage for non-deterministic systems • Create test plans, regression suites, bug triage processes, reproductions, and CI integration for the surrounding product • Monitor live calls after release for drift and degradation and escalate issues to engineers • Red-team agents for prompt injection, off-script behavior, unauthorized identity claims, medical-advice risks, guardrails, and access controls • Test products operated by Vi Operate, which places AI agents into the daily operations of health systems, providers, and pharma organizations; Augment provides the underlying voice capability

🎯 Requirements

• Experience testing a production voice agent that talks to people on the phone; ability to diagnose a real failure and identify the faulty layer • Experience testing a language-model system in production and explaining how its performance was evaluated • Strong classical QA experience, including test strategy, test planning, regression suites, bug triage, and release processes • Experience making non-deterministic output testable and explaining testing tradeoffs • Python proficiency and test automation; ability to build tooling and integrate it into CI • Sound judgment about defect severity and patient-impacting risks • Systems mindset focused on component seams, unexpected inputs, and access-control risks • Remote work aligned with EST/CT hours • Nice-to-have qualifications: • Background in IVR, contact center, or CCaaS quality, including Genesys, NICE, Five9, Amazon Connect, Twilio, or similar • Experience with audio and network condition testing, including codecs, jitter, packet loss, barge-in, and interruption handling • Familiarity with LLM-as-judge harnesses, prompt regression frameworks, or simulated conversation testing • Healthcare or pharma experience and comfort working under HIPAA • Experience establishing a QA function • Load and performance testing on real-time systems • Spanish proficiency sufficient to evaluate agent calls

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume