Associate Director – RWD Data Engineering

🔥 1 minute ago

🇺🇸 United States – Remote

💵 $136.5k - $200.2k / year

⏰ Full Time

🟡 Mid-level

🟠 Senior

🚰 Data Engineer

Where this job comes from

Posted on Eli Lilly and Company's careers page ↗Eli Lilly and Company's careers page, not through a job board

First seen by Remote Rocketship 1 minute ago

Apply Now
Logo of Eli Lilly and Company

Eli Lilly and Company

10,000+ employees

💊 Pharmaceuticals

🏥 Healthcare

🧬 Biotechnology

💰 $6.5M Post-IPO Debt - Eli Lilly on 2024-02

Pharmaceuticals • Healthcare • Biotechnology

Eli Lilly and Company is a multinational pharmaceutical company that researches, develops, manufactures, and markets prescription medicines across multiple therapeutic areas including diabetes, oncology, immunology, neuroscience, and pain. Headquartered in Indianapolis, Indiana, Lilly is known for its extensive R&D, clinical development, and global commercialization of biologic and small-molecule drugs. The company focuses on drug discovery, clinical trials, regulatory approvals, and large-scale pharmaceutical manufacturing and distribution.

📋 Description

• Design, develop, and deploy cloud-native data products and high-throughput Databricks pipelines transforming raw real-world data into analysis-ready assets • Use AI coding assistants and agentic development tools to write, refactor, test, and document Python, SQL, and R code • Deliver RWD data products through requirements, prototyping, production deployment, optimization, and maintenance • Build, optimize, and maintain ETL/ELT ingestion and transformation pipelines for large-scale administrative claims and EHR data • Build and maintain lakehouse data assets using Databricks, Delta Lake, Unity Catalog, and Databricks Workflows • Develop reusable transformation logic using healthcare coding and terminology standards • Optimize SQL and distributed processing workloads for performance and compute cost • Translate analytic and research requirements into reproducible extraction and transformation logic for cohort construction and temporal analysis • Design and operate automated data quality and validation frameworks with monitoring and alerting • Apply Git, pull requests, peer review, automated tests, CI/CD, and Databricks Asset Bundles • Build and deploy interactive analytic applications for scientists, statisticians, and medical teams • Build AI-enabled solutions using governed data products, frontier AI models, and Databricks Model Serving • Collaborate with data scientists, statisticians, and technical partners on requirements and engineering standards • Serve as technical point of contact for assigned data sources and products • Contribute to training materials and an inclusive culture focused on high-quality data

🎯 Requirements

• Bachelor’s in Computer Science, Applied Mathematics, Engineering, Statistics, Information Technology, or Bioinformatics • Minimum 5 years' experience of hands-on data engineering experience with a demonstrated focus on healthcare or life sciences RWD, including direct work with large administrative claims and/or EHR datasets • Minimum of 1 year experience of hands-on, production development on the Databricks platform using Python (PySpark), SQL, and Delta Lake, applied directly to real-world healthcare data at scale • Minimum of 1 year experience designing and implementing data quality and validation checks for large clinical datasets • Minimum of 1 year of experience using AI coding assistants to develop production software, and experience building at least one solution using frontier AI model APIs • Expert SQL optimization on Databricks SQL and Spark SQL • Python fluency, including pandas, PySpark, and Polars • Workflow orchestration with Apache Airflow, Prefect, or Dagster, including Databricks Workflows • Distributed computing with Apache Spark, Dask, or Ray • Hands-on production development on Databricks, including notebooks, Workflows, Delta Live Tables, Unity Catalog, SQL warehouses, and cluster configuration • Experience with AWS or Azure object storage and services • Delta Lake or Apache Iceberg experience • ETL/ELT tooling such as AWS Glue, dbt, or equivalent • Daily use of Git and GitHub/Azure DevOps; CI/CD pipelines and Docker • Data modeling with star/snowflake schemas and OMOP-compliant structures • Experience building interactive analytic applications with Dash, R Shiny, Streamlit, or Databricks Apps • Advanced R development and production R Shiny dashboards • Experience designing automated data quality frameworks • Deep hands-on knowledge of administrative claims and complex EHR data • Knowledge of healthcare terminology and ontology mapping • OMOP CDM transformation, vocabulary loading, and Achilles/DQD data quality checks • Experience with phenotyping and cohort construction • HIPAA-compliant data handling, de-identification standards, DUA compliance, and audit trail requirements • Expert daily use of AI coding assistants and agentic coding tools • Hands-on experience integrating frontier AI model APIs, including prompt engineering, tool use/function calling, structured outputs, RAG, and agentic workflows • Experience evaluating AI solutions and implementing guardrails for patient-level data • Familiarity with MLOps tooling such as MLflow, Databricks Model Serving, and Vector Search • Master’s degree, Databricks certification, portfolio of code, and additional listed preferences are preferred or additional preferences

🏖️ Benefits

• Company bonus depending, in part, on company and individual performance • Company-sponsored 401(k) • Pension • Vacation benefits • Medical, dental, vision and prescription drug benefits • Flexible benefits, including healthcare and/or dependent day care flexible spending accounts • Life insurance and death benefits • Time off and leave of absence benefits • Well-being benefits, including employee assistance program, fitness benefits, and employee clubs and activities

Apply Now

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Similar Jobs

🔥 1 hour ago

Airbnb

5001 - 10000

✈️ Travel

📦 Logistics

🛍️ eCommerce

Senior data engineer building Airbnb’s enterprise datasets for AI costs, Unit Economics, and financial metrics. Designing reliable pipelines and data models across large-scale systems.

🔥 4 hours ago

Olsson

1001 - 5000

🏗️ Construction

Fire Protection Engineer leading fire protection design for Olsson’s hyperscale data centers and secure facilities. Providing technical leadership across multidisciplinary engineering projects.

🔥 4 hours ago

Block

10,000+ employees

💼 Consulting

📣 Marketing

💳 Fintech

Sales compensation systems engineer building commission pipelines, data models, and calculation logic. Supporting Block’s Square, Cash App, Afterpay, and other financial technology brands.

🔥 5 hours ago

Highmark Health

10,000+ employees

🛡️ Insurance

💼 Consulting

📦 Logistics

Data Engineer designing scalable pipelines, warehouses, governance, and security for Highmark’s healthcare data infrastructure. Collaborating across teams to deliver reliable analytics solutions.

🔥 5 hours ago

Prompt Therapy Solutions Inc

11 - 50

🏥 Healthcare

💼 Consulting

📦 Logistics

Clinical technical support specialist for Prompt’s rehab therapy SaaS platform. Supporting customers and coordinating accurate clinical data migrations.

Developed by Lior Neu-ner. I'd love to hear your feedback — Get in touch via DM or support@remoterocketship.com
Follow Remote Rocketship on LinkedInFollow Remote Rocketship on InstagramFollow Remote Rocketship on FacebookFollow Remote Rocketship on TikTok