Senior Data Engineer – Azure

🕒 July 10

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Software Mind

WebsiteLinkedInAll Job Openings

1001 - 5000 employees

Founded 1999

🤖 Artificial Intelligence

☁️ SaaS

📡 Telecommunications

💰 Private Equity Round on 2020-12

Artificial Intelligence • SaaS • Telecommunications

Software Mind is a technology company that specializes in software development and digital transformation services. With a focus on AI and cloud solutions, the company offers a wide range of services including custom software development, mobile app development, and cloud consulting. Software Mind serves various industries such as financial services, telecom, biotech, and media, providing tailored solutions to accelerate digital transformations and business growth globally.

📋 Description

• Develop reusable, metadata-driven data pipelines using Databricks Lakehouse architecture and PySpark • Design and implement comprehensive data models • Build robust ETL/ELT solutions with advanced features: Merge operations, SCD Type 2 implementations, etc. • Implement incremental data loads with idempotency patterns and overlap joins optimization • Automate and optimize data platform processes with focus on performance and reliability • Build integrations with data sources and consumers using event-driven patterns • Cooperate with infrastructure engineering team to set up cloud resources • Initiate and implement improvements to data platform architecture

🎯 Requirements

• Databricks expertise: proficient in Databricks Lakehouse architecture and PySpark development • Data modeling mastery: extensive experience in data model definition including fact tables, dimension tables, measures, grain analysis, SCD Type 2, surrogate keys, late-arriving records handling, overlap joins, and incremental loads with idempotency • Programming: advanced Python and PySpark skills for ETL/ELT development • Databricks optimization: deep knowledge of optimize, zOrder, Liquid clustering, ACID transactions and performance tuning • Event-based architecture: proven experience in designing and implementing event-driven data solutions • Azure data platform: experience working with Azure-based datasets and data pipelines • SQL proficiency: strong SQL skills for complex data transformations • Large-scale data processing: experienced in handling large and complex datasets efficiently • CI/CD: experience in developing automated deployment pipelines • Networking fundamentals: understanding of basic networking concepts • Agile methodology: familiar with Scrum and agile development practices

🏖️ Benefits

• Flexible employment and remote work • International projects with leading global clients • International business trips • Non-corporate atmosphere • Language classes • Internal & external training • Private healthcare and insurance • Multisport card • Well-being initiatives