Senior Data Engineer II

🔥 0 minutes ago

🌽 Illinois – Remote

🌽 Illinois – Remote

💵 $90.7k - $153.9k / year

⏰ Full Time

🟠 Senior

🚰 Data Engineer

🦅 H1B Visa Sponsor

This company has sponsored H1B visas in the past.

👻 Ghost score 1%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Allstate

WebsiteLinkedInAll Job Openings

10,000+ employees

Founded 1931

💼 Consulting

📦 Logistics

🛡️ Insurance

💰 Post-IPO Equity on 2014-01

Consulting • Logistics • Insurance

Allstate is an industry leader in providing insurance solutions, focusing on home, auto, device, and identity protection. With a commitment to customer well-being, Allstate aims to instill peace of mind and financial security for its customers. The company also emphasizes community impact and sustainability through various initiatives, showcasing their dedication to social responsibility and positive change.

📋 Description

• Design, build, and maintain scalable batch and real-time data pipelines using Apache Spark, SQL, Python, and cloud-native data technologies • Architect and implement Medallion Lakehouse data patterns for enterprise analytics, reporting, operational intelligence, and AI use cases • Develop and optimize ETL/ELT pipelines from diverse enterprise source systems • Design and implement dimensional models including fact and dimension tables • Build and manage data workloads within Microsoft Fabric, OneLake, and modern Lakehouse platforms • Establish data quality, observability, and governance frameworks with automated validation, reconciliation, monitoring, and alerting • Optimize Spark workloads for performance, scalability, reliability, and cost efficiency • Manage schema evolution, data lineage, metadata, and source-system changes • Develop reusable frameworks, accelerators, and engineering standards • Leverage Agentic AI-assisted development for code generation, testing, documentation, troubleshooting, and productivity • Implement CI/CD and DevSecOps practices for automated testing, deployment, release management, and rollback • Lead technical design reviews and mentor junior and mid-level data engineers • Collaborate with architects, stakeholders, analytics engineers, and data scientists to translate business requirements into scalable solutions • Partner with platform, security, privacy, and governance teams to ensure compliance • Drive continuous improvement across data engineering practices, architecture standards, and Agile delivery processes • Serve as a technical leader and trusted advisor • Champion modern data architecture, reusable standards, and automation-first approaches • Drive adoption of AI-assisted engineering practices • Provide technical leadership, mentorship, and guidance without direct people management responsibilities

🎯 Requirements

• 7+ years of experience as a Data Engineer building and supporting enterprise-scale data platforms and pipelines • Strong hands-on experience with Apache Spark • Advanced proficiency in Python, SQL, and modern data engineering development practices • Experience designing and implementing Lakehouse architectures and Medallion Architecture patterns • Strong understanding of dimensional modeling, including star schemas, snowflake schemas, fact tables, dimension tables, and semantic data models • Experience with Microsoft Fabric, OneLake, Delta/Parquet formats, or comparable cloud analytics platforms • Expertise in data integration, transformation, and optimization techniques for large-scale analytical workloads • Experience implementing CI/CD pipelines, source control, Infrastructure as Code (IaC), and automated testing practices • Familiarity with AI-powered engineering tools and Agentic AI-assisted software development practices • Strong understanding of distributed computing, performance tuning, scalability, fault tolerance, and data reliability • Experience leading technical initiatives and mentoring engineering team members • Preferred: experience with Kafka, Event Streams, or similar technologies • Preferred: experience designing enterprise data products within Microsoft Fabric • Preferred: familiarity with Data Vault, Kimball, or other enterprise data modeling methodologies • Preferred: experience supporting machine learning, generative AI, or advanced analytics workloads • Preferred: knowledge of data governance, metadata management, master data management, and data quality frameworks • Preferred: cloud or data engineering certifications • Required background investigation • Must have reliable internet with minimum speeds of 50 MB download and 5 MB upload when working from home • Must have a dedicated, private workspace free from distractions with appropriate desk and seating • Allstate generally does not sponsor individuals for employment-based visas for this position

🏖️ Benefits

• Comprehensive technology setup including a laptop, monitors, headset, keyboard, and mouse • Monthly connectivity reimbursement for employees eligible to work from home • Remote work arrangement • Dedicated private workspace and reliable internet requirements support for home working