AWS Data Engineer

Job not on LinkedIn

🔥 1 minute ago

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

VALCE Talent Solutions

WebsiteLinkedInAll Job Openings

11 - 50 employees

Founded 2016

🤝 B2B

🎯 Recruiter

💼 Consulting

B2B • Recruitment • Consulting

VALCE Talent Solutions is a company specializing in nearshoring, IT talent acquisition, and consultancy aimed at helping businesses expand globally, particularly in Mexico, LATAM, and the United States. They offer customized solutions that include talent recruitment, workforce management, and the integration of technology and artificial intelligence to enhance business processes. With a strong focus on strategic consulting, VALCE aims to connect technology, talent, and tangible results, boasting a robust network of IT professionals and a proven track record of successful placements and project scaling.

📋 Description

• Develop reusable Python packages for internal or public usage • Automate ETL processes and schedule jobs using Airflow DAGs • Track pipeline runs and reprocess error records • Orchestrate pipelines to run sequentially or in parallel • Troubleshoot and fix data pipeline errors • Import and export data in CSV, JSON, XML, and other formats, preferably using S3 or other cloud storage • Optimize complex SQL queries for data warehouse technologies, preferably Snowflake • Implement Python unit, integration, and system tests • Implement DBT tests for data validation and quality checks • Generate code from configurations using Python and Jinja templates • Implement GitHub version control and CI/CD processes • Manage AWS data storage and security solutions, including S3, ECS, and IAM • Integrate AWS Lambda, Apache Airflow, DBT models, EMR, S3, Redshift, and Snowflake • Develop API integrations with Salesforce, SFTP, SharePoint, OneDrive, and other systems • Create complex DBT models, macros, snapshots, and documentation • Monitor pipelines and configure alerts for workflow failures using CloudWatch and DataDog • Work as an individual contributor and as part of a team • Communicate with and face clients

🎯 Requirements

• 10+ years of experience • Excellent communication and client-facing skills • Attention to detail • Individual contributor capability and ability to work as part of a team • 100% hands-on experience with the mentioned skills • Advanced proficiency in Python, including code structures, modules, packages, classes, inheritance, multithreading, and functional programming • Experience developing reusable Python packages • Experience automating ETL processes and scheduling jobs with Airflow DAGs • Advanced SQL skills, including complex joins, CTEs, and subqueries • Experience optimizing SQL queries for data warehouse technologies, preferably Snowflake • Proficiency in Python unit, integration, and system testing • Proficiency implementing DBT tests for data validation and quality checks • Experience generating code with Python and Jinja templates • Experience using GitHub and implementing CI/CD processes from scratch • In-depth understanding of AWS S3, ECS, and IAM, including organization and security best practices • Knowledge of AWS security best practices, IAM roles, encryption standards, secure coding guidelines, and DBT profile access configurations • Experience using an AI IDE tool • Nice to have: AWS Lambda experience • Nice to have: Apache Airflow on AWS proficiency • Experience integrating Airflow with AWS services and DBT models • Experience with API integrations for Salesforce, SFTP, SharePoint, OneDrive, and other systems • Nice to have: DBT Core/Cloud proficiency, including complex models, incremental models, snapshots, documentation, and reusable macros • Nice to have: familiarity with AWS CloudWatch and DataDog for pipeline monitoring and alerts