Senior Cloud Data Engineer, GCP Data Fabric

Job not on LinkedIn

🔥 10 minutes ago

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

OmegaHires

WebsiteLinkedInAll Job Openings

11 - 50 employees

🎯 Recruiter

🏢 Enterprise

Recruitment • Enterprise • IT Services

OmegaHires is a full-service digital and enterprise IT staffing firm that focuses on providing both permanent and contingent workforce solutions across various digital and enterprise technologies. The company specializes in UX development with full stack developers using modern technologies such as React, Angular, and Node. js, as well as microservices development utilizing Java, Scala, Go, and SQL. OmegaHires also provides expertise in integration and big data technologies, DevOps for automated CI/CD pipelines, and enterprise IT management including ERP and SAP systems. Committed to "responsible recruiting," OmegaHires uses unique sourcing and assessment methodologies and offers training and mentorship to ensure candidates meet the evolving needs of their clients.

📋 Description

• Lead migration of over 100TB of enterprise data from Cloudera Hadoop environments into Google Cloud Platform • Migrate data from HDFS, Hive, AWS S3, and FSx into BigQuery • Validate data integrity throughout migration activities • Optimize storage, performance, and scalability • Design and develop modern cloud-native data pipelines • Build scalable ingestion and transformation frameworks • Develop optimized BigQuery storage models • Support enterprise Data Fabric implementation • Design and implement Medallion Architecture (Bronze, Silver, Gold) • Develop secure and scalable data movement frameworks • Improve platform reliability, performance, and operational efficiency • Support cloud-native data engineering best practices

🎯 Requirements

• 6+ years of Cloud Data Engineering experience • Advanced experience with Cloudera (HDFS/Hive) • Experience migrating large-scale enterprise data platforms • Strong Google Cloud Platform (GCP) experience • Experience with BigQuery • Experience building modern cloud data pipelines • Experience working with Hadoop ecosystems (HDFS, Hive) • Experience with AWS data storage services (S3 and/or FSx) • Experience designing and implementing Medallion Architecture (Bronze, Silver, Gold) • Strong SQL and data engineering fundamentals • Experience optimizing large-scale data storage and processing • Excellent troubleshooting and problem-solving skills • Data Fabric implementation experience (preferred) • Serverless data architecture (preferred) • Data lake modernization (preferred) • Data governance and security (preferred) • Data quality and validation frameworks (preferred) • CI/CD for data engineering (preferred) • Infrastructure as Code (preferred) • Python or Spark development experience (preferred) • Enterprise cloud migration experience (preferred)