Senior Business Intelligence Analyst – AI Evaluation, Monitoring

🕒 May 6

🔔 Pennsylvania – Remote

🔔 Pennsylvania – Remote

⏰ Full Time

🟠 Senior

📊 Business Intelligence Developer

🦅 H1B Visa Sponsor

This company has sponsored H1B visas in the past.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Geisinger

WebsiteLinkedInAll Job Openings

10,000+ employees

Founded 1915

💊 Pharmaceuticals

🧘 Wellness

Healthcare • Pharmaceuticals • Wellness

Geisinger is a healthcare organization that has been providing accessible medical services for over a century in Pennsylvania. It focuses on meeting the healthcare needs of its communities and is dedicated to innovative patient care. With career opportunities in various fields including nursing, allied health, and administration, Geisinger promotes professional development and a supportive workplace for its employees, emphasizing diversity, equity, and inclusion.

📋 Description

• Serve as a partner to the business or clinical area supported (AI Evaluation & Monitoring) • Use analytic expertise to make data-driven decisions • Collect, clean, and interpret data sets to solve problems • Identify and communicate improvement opportunities • Support evaluation of existing AI/ML projects • Mentor junior BI developers through feedback and training • Independently consult with stakeholders on evaluation and data requirements • Execute the evaluation work for Geisinger's AI portfolio • Collaborate in the design and development of reporting solutions • Recommend analysis approach based on business questions • Develop, validate, and execute complex queries and reports • Present analysis and reports to internal customers • Troubleshoot data quality issues when profiling, testing, or validating

🎯 Requirements

• Minimum 8 years of relevant experience • Healthcare, clinical, or regulated-industry data experience strongly preferred • Applied statistics: hypothesis testing, confidence intervals, power and sample size, regression, and non-parametric methods • SQL with large, messy clinical data warehouses, including window functions, CTEs, debugging query plans, and performance tuning • Python for analysis: pandas, numpy, scikit-learn, statsmodels, common evaluation libraries • Production monitoring design and execution: drift detection, performance decay tracking, adoption and outcome metrics • LLM and generative AI evaluation execution: golden sets, judge-based scoring, hallucination and grounding checks • High School Diploma or Equivalent (GED) - Required

🏖️ Benefits

• Healthcare benefits from day one, including vision and dental • Atmosphere of collaboration, cooperation, and collegiality