Data Engineer – AWS, Python

Job not on LinkedIn

18 hours ago

Apply Now

Receive Emails with Similar Jobs

3Pillar Global

WebsiteLinkedInAll Job Openings

SaaS • Enterprise • Artificial Intelligence

3Pillar Global is a modern application strategy, design, and engineering firm that specializes in delivering strategic software development initiatives for various industries. They offer a range of services, including application technology strategy, digital product engineering, data and analytics, and artificial intelligence development. 3Pillar Global focuses on helping organizations transform their bold ideas into breakthrough solutions by leveraging cutting-edge technologies such as generative and multimodal AI. They work with partners and clients across multiple sectors, including healthcare, financial services, insurance, media, and information services, to solve complex technology challenges and deliver high-performing results.

1001 - 5000 employees

☁️ SaaS

🏢 Enterprise

🤖 Artificial Intelligence

💰 Private Equity Round on 2021-10

📋 Description

• Monitor approximately 2,300 scheduled jobs (daily, weekly, monthly) to ensure timely and successful execution. • Troubleshoot job failures, perform detailed root cause analysis (RCA), and provide clear documentation for all findings. • Address and resolve bugs and data-related issues reported by the business team. • Verify source file placement in designated directories to maintain data integrity. • Help manage synchronization between external databases (including Teradata write-backs) and AWS Glue tables. • Assist in developing new solutions, enhancements, and bug fixes using AWS Glue, Lambda, and Step Functions. • Answer questions from the business and support User Acceptance Testing (UAT) inquiries. • Make timely decisions to resolve issues, execute tasks efficiently, and escalate complex problems to senior or lead engineers as needed. • Participate in knowledge transfer to internal resources and assist in creating necessary documentation and runbooks

🎯 Requirements

• Technical Skills: In-depth knowledge of AWS Glue, AWS Lambda, and AWS Step Functions. • Core Concepts: A deep understanding of ETL processes and data warehouse design. • Problem-Solving: Proven ability to troubleshoot data pipelines and perform root cause analysis (RCA). • Experience: 3-5 years of relevant experience. • Experience: Hands-on experience with Glue, Lambda, and Step Function development. • Availability: Must be able to work a day shift that includes coverage for weekends and holidays on a rotational basis. • Preferred Skills: Familiarity with Change Data Capture (CDC) processes, Teradata, and integration with SAS & RMJ events. AWS Certified.