Senior Data Engineer

🔥 16 hours ago

🌽 Illinois – Remote

🌽 Illinois – Remote

💵 $80k - $136.6k / year

⏰ Full Time

🟠 Senior

🚰 Data Engineer

🦅 H1B Visa Sponsor

This company has sponsored H1B visas in the past.

👻 Ghost score 0%

What it means: the chance that no one is really hiring for this job.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Allstate

WebsiteLinkedInAll Job Openings

10,000+ employees

Founded 1931

💼 Consulting

📦 Logistics

🛡️ Insurance

💰 Post-IPO Equity on 2014-01

Consulting • Logistics • Insurance

Allstate is an industry leader in providing insurance solutions, focusing on home, auto, device, and identity protection. With a commitment to customer well-being, Allstate aims to instill peace of mind and financial security for its customers. The company also emphasizes community impact and sustainability through various initiatives, showcasing their dedication to social responsibility and positive change.

📋 Description

• Design, develop, and enhance scalable data pipelines and data processing systems • Build reusable ingestion, transformation, and storage patterns • Integrate data from APIs, databases, event streams, and third-party systems • Implement data models and schemas creating unified views of business operations • Apply version control, testing, CI/CD, and automated deployments • Ensure data quality, integrity, and reliability through validation, monitoring, and observability • Optimize workflows for performance, cost efficiency, and operational resilience • Document data flows, lineage, and technical components • Collaborate with product managers, engineers, and analysts to understand data requirements • Participate in iteration planning, daily standups, retrospectives, and product ceremonies • Contribute to metadata management, data lineage, privacy, security, and regulatory compliance • Evaluate new technologies, frameworks, and patterns • Share knowledge and mentor less experienced engineers • Own the end-to-end lifecycle of data products from design and implementation through production support, data quality, and performance metrics

🎯 Requirements

• Strong proficiency in developing scalable data pipelines using PySpark and Python • Hands-on experience with modern data engineering frameworks and tools, including Apache Spark, Kafka, Flink, dbt, or equivalent technologies • Strong knowledge of SQL, NoSQL databases, data modeling, and data architecture principles • Demonstrated ability to optimize SQL queries, data pipelines, and storage for performance, scalability, and cost efficiency • Experience designing and developing batch and/or real-time streaming data solutions • Hands-on experience with Microsoft Fabric Notebooks for developing, debugging, testing, and operationalizing data engineering workflows • Experience with containerization and orchestration technologies, such as Docker and Kubernetes • Experience implementing monitoring and observability solutions; Datadog experience preferred • Ability to work effectively in collaborative, iterative, and product-centric engineering environments • Strong communication, analytical, troubleshooting, and problem-solving skills • Experience with cloud data services (Azure, AWS) preferred • Hands-on experience with Microsoft Fabric components, including Lakehouse, Warehouse, Data Pipelines, Dataflows Gen2, and Semantic Models preferred • Familiarity with CI/CD pipelines, such as Jenkins, GitHub Actions, and Azure DevOps preferred • Familiarity with generative and agentic AI tools preferred • Must submit to a background investigation • Allstate generally does not sponsor individuals for employment-based visas for this position • Dedicated, private workspace free from distractions with appropriate desk and seating • Reliable internet with minimum speeds of 50 MB download and 5 MB upload

🏖️ Benefits

• Comprehensive technology setup including a laptop, monitors, headset, keyboard, and mouse • Monthly connectivity reimbursement to help offset internet costs • Remote work arrangement

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume