Azure Databricks Engineer

Emploi pas sur LinkedIn

🕒 il y a 14 jours

Postuler Maintenant

Trouver des Emplois à Distance Similaires

Recevoir des emails pour des emplois à distance

📊 Vérifiez votre score de CV pour ce poste

Améliorez vos chances d'obtenir un entretien en vérifiant votre score de CV avant de postuler.

Télécharger le CV

opinov8

Site WebLinkedInTous les Emplois

201 - 500 employés

Fondée en 2017

🤝 B2B

🏢 Entreprise

B2B • Enterprise

opinov8 est une entreprise mondiale de solutions numériques et d'ingénierie, spécialisée dans le développement de logiciels sur mesure, les services cloud et les services de données. Basée à Londres, avec des centres de développement et des équipes distantes en Ukraine, en Égypte, en Colombie, au Portugal et dans l'UE, opinov8 collabore avec des entreprises du monde entier pour accélérer leur transformation numérique grâce à une ingénierie personnalisée, des migrations vers le cloud et des solutions axées sur les données. L'entreprise fonctionne en tant que fournisseur de services B2B prioritairement à distance, desservant des clients dans des secteurs tels que la finance, la santé, la logistique et les médias.

Description

• Design, build, and maintain scalable ETL/ELT pipelines using Azure Databricks and Apache Spark; • Develop and optimize PySpark/Spark SQL notebooks for large-scale data processing; • Implement batch and real-time data ingestion pipelines using Azure Data Factory (ADF), Databricks Workflows, and Structured Streaming; • Manage and optimize Delta Lake/Lakehouse architecture, including Bronze, Silver, and Gold layers; • Integrate data from multiple sources including APIs, databases, cloud storage, and third-party systems; • Work with Azure services such as ADLS Gen2, Azure Synapse Analytics, Azure SQL Database, and Azure DevOps; • Monitor, troubleshoot, and improve pipeline performance, scalability, and cost optimization; • Implement data governance, security, access controls, and compliance standards using Unity Catalog and RBAC; • Collaborate with Data Engineers, Data Scientists, Analysts, and Product teams to deliver reliable data solutions; • Support CI/CD implementation, automation, and infrastructure-as-code practices.

🎯 Exigences

• Bachelor’s degree in Computer Science, Engineering, Information Technology, or a related field • 5+ years of experience in Data Engineering or Big Data development • Strong hands-on experience with Azure Databricks and Apache Spark • Proficiency in PySpark, Spark SQL, and SQL • Experience with Azure Data Factory (ADF) and Azure Data Lake Storage (ADLS) • Knowledge of Delta Lake, Lakehouse architecture, and data modeling concepts • Experience with distributed data processing and performance tuning • Familiarity with CI/CD pipelines, Git, and Azure DevOps. • NICE TO HAVE: Experience with Kafka, dbt, Airflow, or Snowflake. Knowledge of Terraform or Infrastructure as Code (IaC). Exposure to MLflow, machine learning pipelines, or MLOps.

🏖️ Avantages

• Digital-First Approach: Great talent knows no borders! You can work from wherever you are — we hire and collaborate with professionals worldwide. • Remote Work Model: Balance your professional and personal life with our flexible working conditions, empowering you to deliver your best from anywhere. • Exciting Projects: Dive into impactful projects across industries that challenge and spark creativity. • Boost Your Expertise: Grow your career with continuous learning, development opportunities, and hands-on experience. • Join the Best Team Ever: Collaborate with our diverse and cross-cultural team of passionate technologists and creative thinkers.