Clinical AI Evaluation Specialist

Stelle nicht auf LinkedIn

🕒 vor 1 Monat

Jetzt Bewerben

Ähnliche Remote-Jobs finden

E-Mails für Remote-Jobs erhalten

📊 Überprüfen Sie Ihre Lebenslauf-Bewertung für diese Stelle

Verbessern Sie Ihre Chancen auf ein Vorstellungsgespräch, indem Sie Ihre Lebenslauf-Bewertung vor der Bewerbung überprüfen.

Lebenslauf hochladen

CentralReach

WebsiteLinkedInAlle Stellen

201 - 500 Mitarbeiter

Gegründet 2010

💰 Private equity im 2018-03

CentralReach bietet die führende Software- und Dienstleistungsplattform für Autismus- und IDD-Pflege, um Kindern und Erwachsenen mit Autismus sowie verwandten intellektuellen und entwicklungsbezogenen Behinderungen - und denjenigen, die sie betreuen - Potenziale zu erschließen, bessere Ergebnisse zu erzielen und ein unabhängigeres Leben zu führen.

Beschreibung

• Contribute to evaluation frameworks for clinical AI products, including defining acceptance criteria, test plans, clinically relevant rubrics, and performance benchmarks. • Conduct structured sampling reviews of AI-generated outputs, assessing across criteria alignment with ABA principles and BACB ethical standards. • Design and implement monitoring automation in collaboration with engineering, including automated quality checks, alerting thresholds, and drift detection for AI systems. • Contribute to the creation and maintenance of reusable governance artifacts including system cards, monitoring playbooks, evaluation of SOP templates, and risk assessment documentation. • Assist with pre-deployment validation testing for new AI products and features, executing defined testing and results of documentation. • Participate in risk tiering assessments for new and evolving AI products, providing clinical and evaluative perspectives. • Support incident investigation for AI-related issues if/as needed and in alignment with clinical best practice and CR Development Policy. • Collaborate with product management and engineering teams to provide input during product planning and development, ensuring governance considerations are integrated early. • Contribute to the development of evaluation methodology and standards for clinical AI use cases, documenting approaches and refining processes based on findings and emerging best practices. • Contribute to AI literacy and training content for internal teams and customers, drawing on hands-on clinical evaluation experience. • Stay current with developments in AI evaluation methodology, LLM behavior, ABA best practices, and relevant regulatory and ethical standards. • Support CentralReach's thought leadership in responsible AI through contributions to conference presentations, professional publications, or applied research as opportunities arise.

🎯 Anforderungen

• Board Certified Behavior Analyst (BCBA) in good standing • 2+ years of demonstrated experience in quality assurance, evaluation, data analysis, or technical assessment roles, preferably within healthcare or behavioral health settings. • Demonstrated experience with large language models, generative AI, and their applications and limitations in clinical or professional contexts. • Experience with prompt/context engineering, AI output evaluation, or AI workflow design. • Experience with statistical sampling methodologies or healthcare quality assurance frameworks. • Familiarity with AI governance frameworks (NIST AI RMF, EU AI Act risk categories, AMA governance guidance). • Experience developing evaluation rubrics, scoring frameworks, or quality measurement tools. • Background in clinical informatics, research methodology, or systematic quality improvement. • Demonstrated proficiency with data analysis tools and comfort working with quantitative and qualitative evaluation data. • Strong analytical skills with the ability to develop systematic, evidence-based evaluation approaches. • Familiarity with HIPAA compliance, healthcare data standards, and ethical considerations in AI. • Excellent written and verbal communication skills with the ability to document methodology clearly and convey findings to clinical, technical, and leadership audiences. • Demonstrated ability to work collaboratively across clinical, technical, and product teams. • Experience in a remote work environment with strong self-direction and accountability.

🏖️ Vorteile

• Health benefits • Generous PTO • 401(k) matching • Paid parental leave • Hybrid work schedules • Career development support • Wellness programs