AI/ML Evaluation Reviewer – Healthcare AI

Emploi pas sur LinkedIn

🕒 il y a 26 jours

🇺🇸 États-Unis – Télétravail

⏳ Contrat

🟡 Intermédiaire

🟠 Senior

🤖 Intelligence Artificielle

🗣️🇺🇸🇬🇧 Anglais requis

D'où vient cette offre

Publiée sur la page carrières de mpathic ↗la page carrières de mpathic, pas via un site d'emploi

Repérée par Remote Rocketship il y a 26 jours

Toujours ouverte sur le site de mpathic. Vérifiée il y a 2 heures.

Postuler Maintenant
Logo of mpathic

mpathic

11 - 50 employés

Fondée en 2021

🏥 Santé

💼 Conseil

🍽️ Alimentation et boissons

Healthcare • Consulting • Food & Beverage

mpathic est une entreprise spécialisée dans la sécurité et l'analytique alimentée par l'IA, qui propose des produits SaaS et des API pour enregistrer, transcrire, analyser et améliorer les interactions humain–IA et humain–humain dans les environnements cliniques, des sciences de la vie et des entreprises. Leur offre inclut la transcription de la parole en texte, l'anonymisation des données personnelles, l'analytique des comportements et des listes de contrôle, les ensembles de données de référence encadrées par des experts humains, l'expertise en red team, et une API Observing Agent pour ajouter des garde-fous informés par domaine pour les agents de santé et médicaux. mpathic cible les sciences de la vie, les organisations de recherche clinique, les fournisseurs de soins de santé, les assureurs et les équipes RH/entreprises axées sur l'empathie, la sécurité, la réduction des biais et la conformité.

Description

• Independently assess technical evidence supporting healthcare AI applications, including model performance, validation methodology, generalizability, reliability, proposed monitoring, and safeguards • Evaluate model performance and validation methods, reference standards, sample sizes, statistical analyses, performance metrics, operating thresholds, and reported uncertainty • Assess data quality and representativeness, including data leakage, subgroup performance differences, and generalizability across patient populations and healthcare settings • Evaluate technical reliability, robustness, reproducibility, known failure modes, and performance under atypical or out-of-distribution inputs • Review proposed evaluation plans, performance benchmarks, and success criteria for pilot deployment • Assess technical safeguards, escalation mechanisms, and human-in-the-loop processes • Evaluate monitoring and change-management plans, including performance degradation detection, model behavior monitoring, updates, intervention, and rollback criteria • Collaborate with clinicians and other technical experts on cross-disciplinary concerns • Document strengths, limitations, missing evidence, and technical concerns using a standardized review framework • Provide clear, evidence-based recommendations and identify additional evidence or changes needed • Participate in reviewer calibration and quality assurance activities • Maintain independence and confidentiality while following conflict-of-interest, confidentiality, and data-handling requirements

🎯 Exigences

• At least 3 years of professional or applied research experience in machine learning, data science, biostatistics, clinical informatics, or a closely related field, with direct experience evaluating or validating predictive models or AI systems • Demonstrated understanding of model evaluation methodologies, including appropriate performance metrics, validation design, statistical uncertainty, threshold selection, and common sources of bias or methodological error • Experience assessing model performance on independent or held-out data and interpreting whether reported performance generalizes beyond the evaluation dataset • Hands-on experience evaluating at least one relevant class of AI systems, such as clinical natural language processing, large language models, conversational AI, medical imaging, computer vision, or tabular predictive models • Ability to critically assess technical research, evaluation reports, or validation studies and identify methodological limitations or unsupported conclusions • Strong analytical writing skills, including the ability to communicate complex findings clearly to non-technical audiences • Ability to work independently while contributing to multidisciplinary expert review and calibration processes • Willingness to complete confidentiality agreements, conflict-of-interest disclosures, and reviewer training requirements • Experience evaluating AI or ML systems used in healthcare, clinical decision support, digital health, or other high-stakes environments • Familiarity with clinical AI evaluation and reporting frameworks, such as TRIPOD+AI, STARD-AI, CONSORT-AI, DECIDE-AI, or model cards • Experience evaluating generative AI, LLM-based systems, or autonomous AI agents, including multi-turn behavior, reliability, hallucinations, and technical safeguards • Experience with robustness testing, adversarial evaluation, red teaming, or systematic failure-mode analysis • Knowledge of production monitoring, distribution shift, model drift, version control, and change-management practices • Experience designing or evaluating human-in-the-loop systems or comparing model performance against human expert judgments • Familiarity with FDA guidance for AI-enabled medical devices, clinical decision support, or software as a medical device • Experience conducting independent technical reviews, scientific peer reviews, grant reviews, or formal technology evaluations • Experience collaborating with clinicians, researchers, or regulatory stakeholders on AI evaluation or deployment

🏖️ Avantages

• Contract, per-review opportunity • Reviewer orientation to the evaluation framework and project-specific requirements prior to the first assignment

Postuler Maintenant

📊 Vérifiez votre score de CV pour ce poste

Améliorez vos chances d'obtenir un entretien en vérifiant votre score de CV avant de postuler.

Emplois Similaires

🕒 il y a 27 jours

Volga Partners

1001 - 5000

🤖 Intelligence artificielle

🤝 B2B

🏢 Entreprise

Medical trainees evaluating AI-generated healthcare content for Volga Partners. Reviewing clinical reasoning, verifying evidence, and improving healthcare AI accuracy remotely.

🇺🇸 États-Unis – Télétravail

💵 $15 - $35 / heure

⏳ Contrat

🟡 Intermédiaire

🟠 Senior

🤖 Intelligence Artificielle

🗣️🇺🇸🇬🇧 Anglais requis

🕒 il y a 28 jours

Sigma AI

501 - 1000

💼 Conseil

🏥 Santé

📦 Logistique

Freelance Cantonese-English linguistic annotator performing annotation, transcription, validation, and voice tasks. Supporting Sigma.AI’s global artificial intelligence data-collection and annotation projects remotely.

🇺🇸 États-Unis – Télétravail

⏳ Contrat

🟡 Intermédiaire

🟠 Senior

🤖 Intelligence Artificielle

🗣️🇺🇸🇬🇧 Anglais requis

🕒 il y a 1 mois

Pado

1 - 10

🤖 Intelligence artificielle

⚡ Énergie

☁️ SaaS

Reserved GPU deal broker matching AI compute supply and demand for PADO’s LG-backed marketplace. Structuring pricing, margins, contracts, and capacity commitments for AI-native buyers.

🇺🇸 États-Unis – Télétravail

⏳ Contrat

🟡 Intermédiaire

🟠 Senior

🤖 Intelligence Artificielle

🗣️🇺🇸🇬🇧 Anglais requis

🕒 il y a 1 mois

Heritage Werks

51 - 200

📱 Médias

📣 Marketing

Fractional strategic alliances consultant activating consulting and systems-integrator relationships for Heritage Werks Group’s Authority AI. Building partner-sourced opportunities, pilots, proposals, and revenue.

🇺🇸 États-Unis – Télétravail

⏳ Contrat

🟠 Senior

🔴 Expert

🤖 Intelligence Artificielle

🗣️🇺🇸🇬🇧 Anglais requis

🕒 il y a 1 mois

Invisible Technologies

201 - 500

💼 Conseil

🏥 Santé

📦 Logistique

SMB business operator evaluating bundled and standalone AI assistants for Meridial’s technology clients. Testing real business workflows and documenting reproducible failures across social and commerce tools.

🇺🇸 États-Unis – Télétravail

💵 $40 - $50 / heure

🔥 Financement dans la dernière année

💰 €100 000 000 Series unknown en 2025-10

⏳ Contrat

🟡 Intermédiaire

🟠 Senior

🤖 Intelligence Artificielle

🗣️🇺🇸🇬🇧 Anglais requis

Développé par Lior Neu-ner. J'aimerais beaucoup recevoir vos retours — contactez-moi en DM ou par e-mail support@remoterocketship.com
Follow Remote Rocketship on LinkedInFollow Remote Rocketship on InstagramFollow Remote Rocketship on FacebookFollow Remote Rocketship on TikTok