Senior Data Engineer – Hadoop, AWS

🔥 0 minutes ago

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Stefanini Brasil

WebsiteLinkedInAll Job Openings

10,000+ employees

Founded 1987

💼 Consulting

🏥 Healthcare

📦 Logistics

Consulting • Healthcare • Logistics

Stefanini Brasil is a leading provider of digital transformation solutions, offering a range of services including artificial intelligence, cybersecurity, cloud enablement, and consulting. With over 35 years of experience, the company focuses on integrating innovative technologies to help organizations enhance their operations and customer experiences across various industries. Their expertise extends to sectors like healthcare, retail, and industrial goods, enabling businesses to optimize processes and drive value through technology.

📋 Description

• Gather, document, and prioritize jobs, pipelines, and HQL queries in Cloudera, mapping dependencies, data volumes, SLAs, and Shell Script routines. • Redesign pipelines for AWS (Athena, Glue, EMR, Step Functions, Lambda), defining data modeling in Amazon S3 (raw, trusted, refined zones) and Amazon Redshift. • Implement ETL pipelines in AWS Glue with code versioning, applying automation via Infrastructure as Code (Terraform or CloudFormation). • Ensure best practices for partitioning, cataloging (Glue Data Catalog), optimized formats (Parquet/Iceberg), and rigorous pre-production data parity validation (reconciliation). • Lead cutover windows, manage rollback plans, provide post-migration hypercare support, and document runbooks for knowledge transfer.

🎯 Requirements

• Proven hands-on experience migrating HQL queries/routines to AWS (Athena, Glue, or EMR). • Solid, real-world experience in production environments with Cloudera, Hadoop, and Hive. • Advanced proficiency in Shell Scripting for automation and process orchestration. • Experience developing with Terraform or CloudFormation and working with CI/CD pipelines. • Knowledge of access management using IAM (least privilege), AWS Lake Formation, data encryption, and compliance with LGPD (Brazilian General Data Protection Law). • Ownership mindset to take end-to-end operational responsibility, autonomy for technical decision-making, and experience working in agile teams (Scrum/Kanban).

🏖️ Benefits

• Meal allowance or meal voucher • Discounts on courses, universities, and language schools • Stefanini Academy — online platform with free, up-to-date courses and certificates • Mentoring • Discount program for medical consultations and exams • Health insurance • Dental insurance • Discount programs and benefits at partner establishments • Travel club • Pet care benefits