Staff AI Engineer, QA

🔥 0 minutes ago

Where this job comes from

Posted on Harris Computer's careers page ↗Harris Computer's careers page, not through a job board

First seen by Remote Rocketship 0 minutes ago

Apply Now
Logo of Harris Computer

Harris Computer

10,000+ employees

Founded 1993

🏥 Healthcare

💼 Consulting

📦 Logistics

Healthcare • Consulting • Logistics

Harris Computer is a company that provides mission-critical software solutions across various industries, including the public sector, healthcare, utilities, and private sector verticals. They focus on acquiring businesses with growth potential, managing them effectively, and building them for future success. Harris Computer serves over 100,000 customers with a global presence in North America, Europe, Asia, and Australia, employing over 12,000 people. Their mission is to empower customers to deliver on their goals by offering tailored software and services.

📋 Description

• Build the core AI system end-to-end, from the service layer and data pipelines to the evaluation infrastructure, and own its architecture • Design and build evaluation infrastructure, including dataset management, automated test harnesses, and replay of AI systems against historical workflows • Build and maintain evaluation datasets with clinician-reviewed correct answers, ensuring they evolve as systems change • Create error metrics weighted by severity, prioritizing mistakes with clinical or financial impact • Build production monitoring for accuracy, overrides, and safety issues, with actionable alerts for the engineering team • Write adversarial tests for AI systems, including hidden or malicious instructions in documents and invented content • Work with product and clinical leads to define accuracy thresholds and criteria for expanding or pulling back AI capabilities • Mentor a talented team • Establish quality standards for testing intelligent systems impacting patient care

🎯 Requirements

• 7 or more years in software or machine learning engineering, with at least 3 years focused on evaluation, testing, or quality for AI systems • A track record of designing and building complete systems from scratch, taking them from prototype to production • A track record of building and running evaluation systems that teams use in production, not just for one-off analyses • At least 2 years of experience evaluating large language model (LLM) systems in a production environment • Strong statistical skills for measuring accuracy, including sampling, confidence intervals, and error analysis • 1 year + of experience designing evaluation sets for document extraction, classification, or structured output • Deep knowledge of explainable AI (XAI) techniques and AI-powered tools (e.g., SHAP, LIME, Claude) • Clear written communication, including design documents, and the ability to explain complex trade-offs to clinicians, engineers, and executives • Curiosity and willingness to actively adopt and leverage emerging AI tools • Comfortable using a range of AI-enabled tools, such as copilots, chat-based AI, and automation solutions • Experience working within the healthcare or another highly regulated industry • Familiarity with AI-specific testing techniques like metamorphic testing, adversarial testing, or data validation with tools like Great Expectations • A history of mentoring engineers and influencing technical direction across multiple teams

🏖️ Benefits

• Plenty of opportunities to grow your career • Comprehensive medical, dental, and vision benefits • 3 weeks of vacation plus 5 personal days to recharge • Employee stock ownership, RRSP program, 401k matching • A chance to give back through community involvement • Flexible work arrangements to suit your lifestyle • Casual work environment • Comprehensive benefit package • Award-winning culture • Opportunity to learn

Apply Now

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Similar Jobs

🔥 1 hour ago

Siemens Healthineers

10,000+ employees

🏥 Healthcare

⚕️ Healthcare Insurance

🧬 Biotechnology

Senior quality engineer leading post-market investigations and regulatory compliance for Siemens Healthineers’ laboratory diagnostics and informatics products. Advancing patient safety across a global healthcare technology portfolio.

🔥 9 hours ago

Solutionreach, Inc.

501 - 1000

🏥 Healthcare

📣 Marketing

☁️ SaaS

Quality Engineer automating Java, Playwright, and TypeScript tests for Solutionreach, a healthcare technology company. Supporting manual testing, security, compliance, and performance quality.

🔥 15 hours ago

GN Group

5001 - 10000

🏭 Manufacturing

📡 Telecommunications

Mobile QA Engineer testing iOS and Android applications connected to Jabra Enhance hearing aids. Supporting compliant releases and reliable digital hearing experiences for GN.

🔥 20 hours ago

Satsuma Pharmaceuticals, Inc.

11 - 50

💊 Pharmaceuticals

🧬 Biotechnology

Quality Assurance Manager overseeing GMP quality activities, investigations, suppliers, and commercial manufacturing for Satsuma Pharmaceuticals’ Atzumi™ migraine therapy.

🔥 20 hours ago

Natera

1001 - 5000

🏥 Healthcare

🧬 Biotechnology

⚕️ Healthcare Insurance

Quality Engineer supporting Natera’s cfDNA genetic testing laboratories and diagnostic products. Managing audits, CAPA, nonconformances, suppliers, and regulatory quality compliance.

Developed by Lior Neu-ner. I'd love to hear your feedback — Get in touch via DM or support@remoterocketship.com
Follow Remote Rocketship on LinkedInFollow Remote Rocketship on InstagramFollow Remote Rocketship on FacebookFollow Remote Rocketship on TikTok