ETL Engineer
August 30
Description
• Collaborate closely with data scientists to understand the nature of the data being pulled from various sources, identifying key data validations and enrichment processes. • Design and implement automated workflows to extract raw data from diverse sources and ingest it into our datalake environment. • Structure and transform the raw data using advanced data manipulation techniques to ensure consistency, integrity, and usability. • Enrich the data with contextual information and metadata to enhance its value for use by data scientists and analysts. • Work closely with the technical team to optimize performance and scalability of the ELT process. • Ensure data quality and integrity through rigorous validation and testing procedures. • Monitor and troubleshoot data pipeline issues to ensure smooth and uninterrupted operation.
Requirements
• Bachelor's degree in Computer Science, Engineering, or a related field. Advanced degree preferred. • Proven experience designing and implementing ELT processes in a data-intensive environment. • Strong proficiency in data manipulation and transformation using tools such as Python (Pandas), SQL, and ETL frameworks. • Experience working with cloud-based data storage and processing platforms, preferably AWS services such as S3 and DocumentDB. • Familiarity with data governance best practices and regulatory requirements. • Excellent analytical and problem-solving skills, with a keen attention to detail. • Effective communication skills and ability to collaborate with cross-functional teams. • Proactive mindset with a strong desire for continuous learning and improvement. • Experience consuming data from APIs and web scraping
Apply NowSimilar Jobs
August 14
Blanc Labs
51 - 200
Develop ETL processes for data integration and transformation using Azure Data Factory.
August 12
MUTT DATA
51 - 200
Help construct systems and present innovative solutions using data and machine learning.
