Senior Data Engineer Consultant – Data Bricks, Technology Consulting

Job not on LinkedIn

November 15

Apply Now

Receive Emails with Similar Jobs

EY

WebsiteLinkedInAll Job Openings

Finance • Consulting • Technology

EY is a global professional services firm, widely recognized for providing audit, tax, consulting, and advisory services. It focuses on helping its clients solve complex problems and transform their businesses using data and technology. EY is committed to building a better working world by enhancing trust in financial markets and economies globally. Its services include corporate finance advisory, transaction strategy, technology consulting, and more, catering to sectors such as health, energy, finance, government, and more. EY also emphasizes sustainability and innovation in its solutions.

10,000+ employees

Founded 1989

💸 Finance

📋 Description

• Design and implement enterprise-scale data architectures, including data lakes, warehouses, and real-time streaming platforms. • Develop and maintain ETL/ELT pipelines that efficiently process large volumes of structured and unstructured data from diverse sources. • Ensure data quality, governance, and security standards are embedded throughout all data engineering processes. • Hands-on development of Databricks notebooks using PySpark, Python, and SQL for ETL automation. • Create and optimise PySpark scripts for efficient data extraction, transformation, and loading from large datasets. • Implement custom data manipulation, validation, and error handling solutions to enhance ETL robustness. • Provide technical mentorship to junior data engineers and analysts. • Lead code reviews, establish best practices, and drive adoption of modern data engineering tools and methodologies. • Collaborate with cross-functional teams including data scientists, analysts, and software engineers to deliver integrated solutions. • Monitor and optimise data pipeline performance, implementing solutions for scalability and cost-effectiveness.

🎯 Requirements

• 5-7 years of data engineering experience with AWS-native implementations • Proven experience working as part of a data engineering team on ETL migration projects • Hands-on experience implementing Databricks notebooks using PySpark, Python, and SQL for ETL automation • Demonstrated expertise developing PySpark scripts for efficient data extraction, transformation, and loading from large datasets • Experience utilising Python for custom data manipulation, validation, and error handling in ETL processes • Proficiency employing SQL for complex joins, aggregations, and database operations within Databricks environments • Track record of testing, debugging, and optimising data transformation processes for accuracy and performance • Experience verifying data integrity throughout pipeline stages and resolving troubleshooting issues • Proven ability collaborating with cross-functional teams to align ETL migration tasks with project goals and deadlines • Experience in government sector data projects and compliance requirements (preferred) • Experience in HR domain including workforce analytics, payroll systems, and employee data management (preferred) • Experience providing technical mentorship to junior team members (preferred) • Certified in: AWS Certified Data Engineer - Associate (preferred) • AWS Certified Solutions Architect or AWS Certified Big Data - Specialty (preferred) • Databricks Certified Data Engineer Associate or Professional • Amazon Redshift or Snowflake certifications • Agile/Scrum Master certification (preferred)

🏖️ Benefits

• Health insurance • 401(k) matching • Flexible work hours • Paid time off • Remote work options