L1 Data Engineer

Likely ghost job

🕒 May 24

🌐 Egypt, India, +4 more countries – Remote

infoinfo

⏰ Full Time

🟡 Mid-level

🟠 Senior

🚰 Data Engineer

👻 Ghost score 69%

infoinfo
Apply Now
Find Similar Remote Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Logo of DeepSource GmbH

DeepSource GmbH

1 - 10 employees

🤖 Artificial Intelligence

💼 Consulting

Artificial Intelligence • Consulting

DeepSource GmbH is a trusted partner for businesses seeking cutting-edge AI services, specializing in computer vision, natural language processing, and predictive analytics. The company focuses on Arabic NLP and ChatGPT bot development, empowering organizations with innovative solutions to streamline operations and enhance user experiences. DeepSource is dedicated to addressing diverse AI needs by hiring top talent, managing AI projects, and providing tailored consulting and training programs. Their expert team leverages extensive knowledge in AI technologies to create and deploy advanced solutions across multiple sectors, ensuring businesses remain competitive in the digital landscape.

📋 Description

• Design, develop, and maintain scalable data pipelines and ETL/ELT workflows for business intelligence and analytics use cases. • Build and optimize data ingestion processes using Azure Data Factory and Databricks. • Ensure data quality and consistency across all layers of the data platform. • Transform and process large datasets using PySpark and Python. • Write and optimize complex SQL queries for analytical reporting and data validation. • Collaborate with data architects and senior engineers to implement and maintain data models. • Monitor, troubleshoot, and resolve pipeline failures and data quality issues. • Apply root-cause analysis to prevent recurrence of pipeline and data quality issues. • Contribute to documentation of data pipelines, data dictionaries, and engineering standards. • Support evaluation of new tools and approaches to improve data infrastructure.

🎯 Requirements

• 3+ years of professional experience in a Data Engineering or closely related role. • Strong proficiency in Python for data processing, transformation, and automation tasks. • Hands-on experience with Pandas for data manipulation and PySpark for distributed data processing. • Practical experience with Databricks, including notebook development, clusters, and job orchestration. • Experience building and managing data pipelines with Azure Data Factory. • Working knowledge of Azure Synapse Analytics, particularly Spark pool integration. • Solid SQL skills, including query writing, optimization, and performance tuning. • Familiarity with data engineering principles including incremental loading, data lake architecture, and Delta Lake. • Understanding of data governance and security concepts within a cloud data platform. • Experience with SQL Server migration projects, including schema conversion and data movement. • Exposure to Terraform for Azure infrastructure provisioning and management. • Familiarity with CI/CD practices applied to data engineering workflows. • Experience with Delta Sharing or Lakehouse Federation concepts. • Candidates are expected to hold or be actively working toward the Databricks Certified Data Engineer Associate certification.

Apply Now