Senior Business Intelligence Analyst – AI Evaluation, Monitoring

🕒 May 6

🔔 Pennsylvania – Remote

🔔 Pennsylvania – Remote

⏰ Full Time

🟠 Senior

📊 Business Intelligence Developer

🦅 H1B Visa Sponsor

This company has sponsored H1B visas in the past.

Apply Now

Find Similar Remote Jobs

Receive Emails For Remote Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Geisinger

WebsiteLinkedInAll Job Openings

10,000+ employees

Founded 1915

💊 Pharmaceuticals

🧘 Wellness

Healthcare • Pharmaceuticals • Wellness

Geisinger is a healthcare organization that has been providing accessible medical services for over a century in Pennsylvania. It focuses on meeting the healthcare needs of its communities and is dedicated to innovative patient care. With career opportunities in various fields including nursing, allied health, and administration, Geisinger promotes professional development and a supportive workplace for its employees, emphasizing diversity, equity, and inclusion.

📋 Description

• The Senior BI Analyst serves as a partner to the business or clinical area supported (AI Evaluation & Monitoring) • Use analytic expertise including knowledge of data structures, programming, and statistics with critical thinking • Collect, clean, and interpret data sets to answer a question or solve a problem • Identify and effectively communicate improvement opportunities and support evaluation of existing AI/ML projects • Act as the analytics expert in meetings and discussions with stakeholders • Consult with AI stakeholders to gather evaluation and data requirements and define scope on projects of all sizes • Execute the evaluation work that keeps Geisinger's AI portfolio safe, performant, and accountable • Collaborate with internal partners and organizations in design and development of reporting solutions and analytic studies • Develop, validate, and execute complex queries and reports that answer business questions for internal customers • Present analysis and reports to internal customers and stakeholders • Report and help troubleshoot data quality issues during data profiling, testing, or validation • Participate in strategic departmental initiatives and mentor junior BI developers

🎯 Requirements

• Minimum of 8 years-Relevant experience* (Required) • Healthcare, clinical, or regulated-industry data experience strongly preferred • Applied statistics: hypothesis testing, confidence intervals, power and sample size, regression, and non-parametric methods • SQL with large, messy clinical data warehouses, including window functions, CTEs, debugging query plans, and performance tuning • Python for analysis: pandas, numpy, scikit-learn, statsmodels, common evaluation libraries • Production monitoring design and execution: drift detection, performance decay tracking, adoption and outcome metrics • LLM and generative AI evaluation execution: golden sets, judge-based scoring, hallucination and grounding checks • Education: High School Diploma or Equivalent (GED)- (Required)

🏖️ Benefits

• Healthcare benefits for full time and part time positions from day one, including vision, dental and domestic partners • Encouragement of collaboration, cooperation and collegiality