Senior Technical Specialist – Databricks

🔥 0 minutes ago

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Datavail

WebsiteLinkedInAll Job Openings

1001 - 5000 employees

Founded 2007

Cloud • Data Management • Analytics

Datavail is a comprehensive IT solutions provider specializing in analytics, application development, database administration, and cloud services. Their offerings span advanced analytics consulting, cloud migration and management, and application modernization, ensuring clients can leverage their data for competitive advantage. Datavail focuses on delivering tailored solutions to enhance data management and operational efficiency across various sectors.

📋 Description

• Design and build conformed dimensions and transactional fact tables for enterprise reporting. • Develop and maintain data pipelines on Azure Databricks using PySpark, Spark SQL, and Delta Lake. • Implement best practices in dimensional modeling, including: • - Star schemas • - Conformed dimensions • - Surrogate and deterministic hash keys (e.g., SHA-256) • - Transactional fact design • Translate existing SQL Server and stored procedure logic into efficient Spark-based transformations. • Implement incremental data processing and change data capture (CDC) pipelines. • Ensure data reconciliation with source systems to meet accuracy and audit standards. • Collaborate closely with a senior Databricks lead to deliver core data assets within defined timelines.

🎯 Requirements

• 8+ years of experience in data engineering, with strong focus on Azure Databricks. • Proficiency in: • - PySpark • - Spark SQL • - Delta Lake • Deep expertise in dimensional modelling and data warehousing concepts. • Experience with: • - Building scalable production pipelines • - Incremental data loads and CDC patterns • - Performance optimization in distributed systems • Strong understanding of relational databases and ability to interpret complex SQL/stored procedures.

🏖️ Benefits

• None specified