AI/ML Evaluation Reviewer – Healthcare AI

Vaga não está no LinkedIn

🕒 Ontem

🇺🇸 Estados Unidos – Remoto (EUA)

⏳ Contrato/Temporário

🟡 Pleno

🟠 Sênior

🤖 Inteligência Artificial

👻 Score fantasma 25%

O que significa: quão provável esta vaga está aberta sem contratação ativa.Como ler: 60% ou mais significa que achamos que é uma vaga fantasma. Pontuações mais baixas preocupam menos.Como pontuamos: tempo em aberto, republicações, vagas abertas vs tamanho do time, rapidez para fechar vagas, e se o salário está divulgado.Observação: é uma estimativa de dados públicos, não um julgamento da empresa. Vagas reais podem ter pontuação alta, então use como uma dica, não como um fato.

Candidatar-se

Encontrar Vagas Remotas Similares

Configurar Alerta de Email para Novas Vagas

📊 Verifique sua pontuação de currículo para esta vaga

Melhore suas chances de conseguir uma entrevista verificando sua pontuação de currículo antes de se candidatar.

Enviar Currículo

mpathic

SiteLinkedInTodas as Vagas

11 - 50 funcionários

Fundada em 2021

🏥 Saúde

💼 Consultoria

🍽️ Alimentos e Bebidas

Healthcare • Consulting • Food & Beverage

mpathic é uma empresa de segurança e análise baseada em IA que oferece produtos SaaS e APIs para gravar, transcrever, analisar e melhorar interações humano-IA e humano-humano em ambientes clínicos, de ciências da vida e empresariais. Suas ofertas incluem transcrição de fala para texto, redação de PII, análises de comportamento e listas de verificação, conjuntos de dados de referência com curadoria humana, especialização em red-teaming e uma API Observing Agent para adicionar diretrizes informadas por domínio para agentes de saúde e médicos. A mpathic tem como alvo as ciências da vida, organizações de pesquisa clínica, prestadores de serviços de saúde, seguradoras e equipes de RH/empresariais focadas em empatia, segurança, redução de viés e conformidade.

Descrição

• Independently assess technical evidence supporting healthcare AI applications, including model performance, validation methodology, generalizability, reliability, proposed monitoring, and safeguards • Evaluate model performance and validation methods, reference standards, sample sizes, statistical analyses, performance metrics, operating thresholds, and reported uncertainty • Assess data quality and representativeness, including data leakage, subgroup performance differences, and generalizability across patient populations and healthcare settings • Evaluate technical reliability, robustness, reproducibility, known failure modes, and performance under atypical or out-of-distribution inputs • Review proposed evaluation plans, performance benchmarks, and success criteria for pilot deployment • Assess technical safeguards, escalation mechanisms, and human-in-the-loop processes • Evaluate monitoring and change-management plans, including performance degradation detection, model behavior monitoring, updates, intervention, and rollback criteria • Collaborate with clinicians and other technical experts on cross-disciplinary concerns • Document strengths, limitations, missing evidence, and technical concerns using a standardized review framework • Provide clear, evidence-based recommendations and identify additional evidence or changes needed • Participate in reviewer calibration and quality assurance activities • Maintain independence and confidentiality while following conflict-of-interest, confidentiality, and data-handling requirements

🎯 Requisitos

• At least 3 years of professional or applied research experience in machine learning, data science, biostatistics, clinical informatics, or a closely related field, with direct experience evaluating or validating predictive models or AI systems • Demonstrated understanding of model evaluation methodologies, including appropriate performance metrics, validation design, statistical uncertainty, threshold selection, and common sources of bias or methodological error • Experience assessing model performance on independent or held-out data and interpreting whether reported performance generalizes beyond the evaluation dataset • Hands-on experience evaluating at least one relevant class of AI systems, such as clinical natural language processing, large language models, conversational AI, medical imaging, computer vision, or tabular predictive models • Ability to critically assess technical research, evaluation reports, or validation studies and identify methodological limitations or unsupported conclusions • Strong analytical writing skills, including the ability to communicate complex findings clearly to non-technical audiences • Ability to work independently while contributing to multidisciplinary expert review and calibration processes • Willingness to complete confidentiality agreements, conflict-of-interest disclosures, and reviewer training requirements • Experience evaluating AI or ML systems used in healthcare, clinical decision support, digital health, or other high-stakes environments • Familiarity with clinical AI evaluation and reporting frameworks, such as TRIPOD+AI, STARD-AI, CONSORT-AI, DECIDE-AI, or model cards • Experience evaluating generative AI, LLM-based systems, or autonomous AI agents, including multi-turn behavior, reliability, hallucinations, and technical safeguards • Experience with robustness testing, adversarial evaluation, red teaming, or systematic failure-mode analysis • Knowledge of production monitoring, distribution shift, model drift, version control, and change-management practices • Experience designing or evaluating human-in-the-loop systems or comparing model performance against human expert judgments • Familiarity with FDA guidance for AI-enabled medical devices, clinical decision support, or software as a medical device • Experience conducting independent technical reviews, scientific peer reviews, grant reviews, or formal technology evaluations • Experience collaborating with clinicians, researchers, or regulatory stakeholders on AI evaluation or deployment

🏖️ Benefícios

• Contract, per-review opportunity • Reviewer orientation to the evaluation framework and project-specific requirements prior to the first assignment