AWS Lakehouse Data Engineer

🔥 0 minutes ago

🇺🇸 United States – Remote

💵 $113k - $188k / year

⏰ Full Time

🟡 Mid-level

🟠 Senior

🚰 Data Engineer

🦅 H1B Visa Sponsor

This company has sponsored H1B visas in the past.

👻 Ghost score 0%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Guidehouse

WebsiteLinkedInAll Job Openings

10,000+ employees

Founded 2018

🏥 Healthcare

🎖️ Defense

📦 Logistics

💰 Grant on 2023-02

Healthcare • Defense • Logistics

Guidehouse is a global consultancy offering advisory, digital, and managed services across commercial and public sectors. It is purpose-built to support industries such as national security, financial services, healthcare, energy, and infrastructure. Guidehouse collaborates with leaders to navigate complexity and drives transformational changes that impact the future. Their expertise spans data analytics, digital technologies, risk management, and more, with a strong emphasis on sustainability and innovation.

📋 Description

• Design, implement, and operate a cloud-native data platform powering AI/ML, analytics, reporting, and data visualization • Design and implement batch and streaming ingestion from APIs, relational databases, file drops, event streams, and external partners • Implement, test, and optimize Python and PySpark ETL/ELT pipelines for curated analytics-ready datasets • Implement incremental processing, change data capture, data contracts, schema validation, and reusable transformation frameworks • Improve pipeline reliability through automated testing, orchestration, monitoring, retry handling, and operational runbooks • Design and implement a Delta Lakehouse-style platform using AWS-native services • Build and manage a scalable lakehouse on Amazon S3 using Apache Iceberg and Apache Parquet • Implement ACID transactions, schema evolution, partition evolution, snapshot isolation, time travel, and rollback capabilities • Enable fast interactive querying using Amazon Athena, Amazon EMR, AWS Glue, and Amazon Redshift • Optimize performance and cost through partitioning, compaction, file sizing, statistics, caching, lifecycle policies, and compute-storage separation • Establish standardized development, test, and production environments with controlled promotion • Implement governance, metadata management, lineage, fine-grained access control, classification, retention, encryption, and secure data handling • Build operational data quality checks and publish measurable SLAs/SLOs • Implement AWS provisioning with Infrastructure as Code and secure-by-default baselines • Build and enhance CI/CD for data pipelines and lakehouse components, including testing, security checks, deployment, promotion, and rollback • Implement observability with metrics, logs, traces, alerts, dashboards, runbooks, and incident-response procedures • Evaluate and improve platform performance, scalability, reliability, security, and cost • Collaborate with data, application, analytics, AI/ML, security, networking, and cloud platform teams • Maintain architecture diagrams, data models, SOPs, interface specifications, runbooks, and secure configuration baselines • Present technical findings, trade-offs, risks, and recommendations to technical and non-technical stakeholders

🎯 Requirements

• Bachelor's degree in Engineering, Information Technology, Computer Science, Data Engineering, or a related field, or FOUR (4) years equivalent practical experience in lieu of degree • SIX (6) years of relevant experience • Hands-on experience implementing AWS-native data lake or lakehouse architectures using Amazon S3 and services such as AWS Glue, Amazon Athena, Amazon EMR, AWS Lake Formation, and Amazon Redshift • Strong experience developing production ETL/ELT pipelines using Python and PySpark, including data modeling, transformation, testing, performance tuning, and error handling • Hands-on experience with Apache Iceberg, including ACID transactions, snapshots, schema and partition evolution, time travel, table maintenance, and query optimization • Advanced SQL skills and experience supporting analytical queries, semantic layers, reporting tools, and data visualization workloads • Experience implementing metadata management and governance capabilities, including cataloging, lineage, ownership, classification, policy enforcement, and fine-grained access controls • Experience with AWS security fundamentals, including IAM and least privilege, KMS encryption, secrets management, network security, logging, and secure SDLC practices • Experience provisioning AWS resources using IaC and operating data platforms across multiple environments • Experience building or operating CI/CD pipelines for data workflows, including testing, packaging, deployment automation, environment promotion, and rollback • Ability to troubleshoot distributed data-processing workloads and optimize performance, reliability, and cost • Ability to obtain Public Trust clearance

🏖️ Benefits

• Medical, Rx, Dental & Vision Insurance • Personal and Family Sick Time & Company Paid Holidays • Parental Leave • 401(k) Retirement Plan • Group Term Life and Travel Assistance • Voluntary Life and AD&D Insurance • Health Savings Account, Health Care & Dependent Care Flexible Spending Accounts • Transit and Parking Commuter Benefits • Short-Term & Long-Term Disability • Tuition Reimbursement, Personal Development, Certifications & Learning Opportunities • Employee Referral Program • Corporate Sponsored Events & Community Outreach • Care.com annual membership • Employee Assistance Program • Supplemental Benefits via Corestream (Critical Care, Hospital Indemnity, Accident Insurance, Legal Assistance and ID theft protection, etc.) • Position may be eligible for a discretionary variable incentive bonus • Flexible benefits package