AWS Data Engineer, ETL Expert

Job not on LinkedIn

🔥 13 minutes ago

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Tech Minds Agency

WebsiteLinkedInAll Job Openings

1 - 10 employees

Founded 2022

💼 Consulting

📣 Marketing

🛍️ eCommerce

Consulting • Marketing • eCommerce

Tech Minds Agency is a creative IT solutions provider that specializes in helping businesses and individuals leverage technology for growth and success. They offer a diverse range of services including website creation, mobile app development, digital marketing, and training in various technical skills. Their mission is to navigate the evolving tech landscape, enabling clients to enhance their online presence and achieve their business goals. With a focus on personalized service and staying ahead of tech trends, Tech Minds Agency aims to empower its clients through innovative solutions and expert support.

📋 Description

• Build and maintain AWS data pipelines • Develop ETL/ELT pipelines using AWS Glue, PySpark, Python, and SQL • Ingest data from sources like Salesforce, databases, APIs, and S3 • Load curated data into Amazon Redshift and data lake storage • Optimize Athena and data lake performance • Convert JSON/CSV data into Parquet • Use Snappy compression • Design proper partitioning strategies • Manage modern data lake architecture • Work with Apache Iceberg tables • Perform migrations from traditional Parquet tables • Support schema evolution, time travel, and ACID transactions • Production support and troubleshooting • Investigate Glue jobs that suddenly become slow • Debug Lambda timeouts • Fix missing records and data quality issues • Resolve Redshift performance problems • Build AWS infrastructure using Terraform • Create reusable modules • Manage Auto Scaling Groups, ALBs, IAM, Lambda, Secrets Manager, S3, Redshift, and DynamoDB • Configure Lambda-based secret rotation • Implement IAM least-privilege access • Work with enterprise backup tools like Rubrik • Validate backup jobs • Perform restores • Support disaster recovery testing • Validate source and target record counts • Maintain audit/control tables in DynamoDB • Check checksums and duplicate records • Troubleshoot data discrepancies reported by business users • Utilize AWS Services: S3, Glue, Athena, Lambda, Redshift, DynamoDB, EventBridge, Step Functions, CloudWatch, Secrets Manager, IAM, Auto Scaling Groups, Application Load Balancer

🎯 Requirements

• Relevant Experience: 9+ • Mandatory Tech Stack: Source: Salesforce (ECRM & OSC), Backup: Grax, Cloud: AWS (EC2, S3, CloudWatch, DynamoDB, RDS, Secrets Manager, ALB, ASG), Infrastructure as Code: Terraform • File Format: Parquet • Database: Blue Database • Query Language: SQL