Cloud Data Engineer

2 days ago

🏄 California – Remote

Although this job is listed in California, this company may be open to hiring remote in other states.

🐊 Florida – Remote

Although this job is listed in Florida, this company may be open to hiring remote in other states.

+4 more states

🏄 California – Remote 🐊 Florida – Remote 🗽 New York – Remote 🎸 Tennessee – Remote 🤠 Texas – Remote ☕ Washington – Remote

💵 $120k - $140k / year

⏰ Full Time

🟠 Senior

🔴 Lead

🚰 Data Engineer

🦅 H1B Visa Sponsor

Apply Now

Receive Emails with Similar Jobs

GovX

WebsiteLinkedInAll Job Openings

Marketplace • B2C

GovX is an online platform offering exclusive discounts for current and former military personnel, first responders, and law enforcement. Members can access special deals on a wide range of products, events, tickets, travel offers, and participating brands. The marketplace is designed to provide savings to those who serve, supporting them with benefits and an easy-to-use shopping experience. By partnering with various brands, GovX extends significant discounts as a token of appreciation for the services these individuals provide.

51 - 200 employees

🏪 Marketplace

👥 B2C

📋 Description

• Supporting and modernizing existing data integrations. • Crafting and maintaining efficient data pipeline architecture. • Assembling large, complex data sets that meet business requirements. • Create and maintain optimal data pipeline/flow architecture. • Identifying, crafting, and implementing internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. • Partner with business analysts, data scientists, and IT teams to translate business requirements into scalable data solutions using Fabric, Spark, using Delta Lake. Develop Spark notebooks for both batch and streaming ETL pipelines leveraging Delta Lake. • Implement and optimize Delta Lake features including schema enforcement, schema evolution, and time travel for robust data management. Optimize Delta Lake tables for performance using Z-ordering, compaction, and partitioning strategies. • Working with the team to strive for clean and meaningful data, and greater functionality and flexibility within the team’s data systems. • Design processes supporting data transformation, data structures, metadata, dependency, and workload management.

🎯 Requirements

• Hands-on experience developing, debugging, and optimizing Spark notebooks for ETL and analytics in Microsoft Fabric and Azure. • Deep expertise in Microsoft Fabric, Dataflows Gen2, and Power BI integration. • Hands-on experience with Delta Lake table management, including schema evolution, versioning, and data compaction. • Experience with Data Lakehouse and Medallion Architecture. • Experience with CI/CD and version control using Git. • Advanced SQL and NoSQL query authoring; Python and Spark scripting. • Proficiency with object-oriented/object function scripting languages: Python, Spark, etc. • Proficiency with Metadata-Driven Design and JSON. • Experience working with streams such as Event Hubs and Event Driven Architectures. • Experience building, maintaining, and optimizing ‘big data’ data pipelines, architectures, and data sets. • Experience cleaning, testing, and evaluating data quality from a wide variety of ingestible data sources. • Knowledge Microsoft Power Platform including Copilot Studio and Power Apps. • Strong collaboration and communication skills with business and technical teams.

🏖️ Benefits

• Paid Time Off, Paid Sick Leave, Paid Holidays • Competitive Medical, Dental, Vision, Short Term Disability, and Life Insurance • 401(k) plan with discretionary match available • Flexible Spending Account (FSA), Health Savings Account (HSA) • Voluntary benefits including Critical Illness, Group Accident, and Voluntary Life • Employee Referral Program • Exposure to a growing ecommerce company • Discounts on the GOVX website