Senior Data Engineer

🕒 May 17

🌽 Illinois – Remote

🌽 Illinois – Remote

⏰ Full Time

🟠 Senior

🚰 Data Engineer

🦅 H1B Visa Sponsor

This company has sponsored H1B visas in the past.

Apply Now

Find Similar Remote Jobs

Receive Emails For Remote Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

CDW

WebsiteLinkedInAll Job Openings

10,000+ employees

Founded 1984

🏢 Enterprise

☁️ SaaS

🔒 Cybersecurity

💰 Post-IPO Equity on 2015-07

Enterprise • SaaS • Cybersecurity

CDW is a leading provider of technology solutions, offering a comprehensive range of products and services to address the needs of enterprises, government, healthcare, and education sectors. They specialize in providing IT infrastructure services, managed services, and security solutions. CDW partners with top technology brands to deliver customized solutions that enhance business productivity and performance across various industries. With a strong focus on digital transformation, security, and cloud solutions, CDW supports businesses in navigating the complexities of modern IT environments. Their expertise spans several domains, including digital workspaces, cybersecurity, and enterprise solutions, making CDW a key partner in driving technological advancements.

📋 Description

• Build and maintain scalable data pipelines, ETL and ELT processes, and data models within the Databricks platform. • Design, develop, and deploy data and AI solutions using Databricks, Spark, Delta Lake, and related technologies. • Develop batch and streaming pipelines using tools such as Databricks Workflows and Azure Data Factory. • Design logical data flow diagrams and normalized schemas, implementing Lakehouse patterns such as the Medallion Architecture (Bronze, Silver, Gold layers). • Ensure data quality, integrity, security, and governance throughout the data lifecycle, including use of Unity Catalog. • Optimize Spark jobs and data transformations through effective partitioning, caching, and join strategies. • Monitor pipeline execution, identify failures, and troubleshoot complex data processing issues. • Collaborate with data architects, analysts, data scientists, and business stakeholders to understand requirements and deliver solutions. • Support documentation of data processes, standards, and data flows.

🎯 Requirements

• 5 Years of experience designing, developing, and deploying data solutions on the Databricks platform. • Proficiency in Python, including PySpark, and SQL. • Hands-on experience with Spark, Delta Lake, and Lakehouse architectures. • Experience implementing data quality, governance, and security practices across data pipelines. • Familiarity with machine learning concepts, tools, and libraries such as TensorFlow, PyTorch, Scikit-learn, and MLflow is a plus. • Experience configuring and integrating external AI models and working with AI governance and monitoring tools is a plus. • Experience with asynchronous programming patterns in Python for building scalable data or AI workloads is a plus. • Strong problem-solving, collaboration, and communication skills.

🏖️ Benefits

• Health insurance • 401(k) matching • Paid time off • Professional development