Cloud Data Engineer

August 2

Apply Now

Receive Emails with Similar Jobs

Globaldev Group

WebsiteLinkedInAll Job Openings

Building remote teams and providing software development solutions for tech businesses 🇺🇸🇮🇱🇩🇪🇺🇦🇵🇹🇵🇱

Dedicated Development Team • Remote R&D Office • Recruiting On-site • Agile Software Development • Outstaffing

201 - 500

Description

• Manage and orchestrate complex data pipelines using Azure Data Factory (ADF); • Design, implement, and optimize data transformation processes using Python libraries (Pandas, Numpy, PySpark); • Develop, maintain, and troubleshoot scalable data workflows on Databricks; • Utilize Azure Data Explorer (ADX) for real-time data analysis and complex queries; • Create and optimize SQL queries to extract, transform, and load data; • Automate routine data tasks with Automation Runbooks and Azure Functions; • Design and manage scalable data storage solutions using Azure Data Lake; • Write and execute unit tests to ensure the integrity and quality of data; • Utilize CI/CD pipelines using Azure DevOps for seamless integration and deployment; • Optimize Azure usage to manage and reduce costs effectively; • Ensure to work in compliance with Hydro Quality system, HSE regulations, policies and standardized operating procedures; • Perform all other tasks upon the instructions of the superior in charge which may be necessitated by the operations of the related unit and which do not conflict with any applicable laws, statutory provisions and company rules; • Compliance with area-specific customer requirements.

Requirements

• Experience with MS Azure for designing and building analytics pipelines, data platforms, and data lake solutions; • Ability to handle the entire product development lifecycle, from proof of concept (PoC) to production deployment of BI and analytics solutions; • Strong collaboration skills with business and technical teams, including DevOps, ML, and infrastructure engineers, to achieve business objectives; • Ability to represent data product developers and influence data product strategy by working with cross-functional teams; • Experience managing high-profile projects and solving data engineering and ML engineering problems for internal customers. • Microsoft Certified: Azure Data Engineer Associate or similar certification is a plus; • Knowledge of Infrastructure as Code (IaC) tools (e.g., Terraform, ARM templates) is a plus; • Awareness of Azure pricing models and cost management practices is a plus.

Benefits

• Direct cooperation with the already successful and growing project • Truly competitive salary • Help and support from our caring HR team.