Data Engineer, Databricks

🕒 May 12

🇺🇸 United States – Remote

⏰ Full Time

🟡 Mid-level

🟠 Senior

🚰 Data Engineer

🦅 H1B Visa Sponsor

This company has sponsored H1B visas in the past.

Apply Now

Find Similar Remote Jobs

Receive Emails For Remote Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Datavail

WebsiteLinkedInAll Job Openings

1001 - 5000 employees

Founded 2007

Cloud • Data Management • Analytics

Datavail is a comprehensive IT solutions provider specializing in analytics, application development, database administration, and cloud services. Their offerings span advanced analytics consulting, cloud migration and management, and application modernization, ensuring clients can leverage their data for competitive advantage. Datavail focuses on delivering tailored solutions to enhance data management and operational efficiency across various sectors.

📋 Description

• Lead and contribute to end-to-end Databricks implementations for clients, including data migration, Lakehouse architecture, and pipeline development • Gather technical requirements, design solutions, and present recommendations to client stakeholders (technical and business) • Build scalable ETL/ELT pipelines using PySpark, Delta Lake, Delta Live Tables (DLT), and Databricks Workflows • Design and implement Databricks Genie • Design and implement semantic layers • Use Databricks AI features to accelerate development, debugging, and code optimization • Design and implement Lakebase architectures for operational and analytical workloads, including transactional data use cases • Develop solutions using SDLC best practices, including modular code design, testing, and documentation • Use Git based version control with proper branching strategies • Implement CI/CD pipelines for Databricks asset • Implement data quality checks, validations, and expectations within workflows • Design and implement Unity Catalog governance, security, and lineage solutions • Optimize Databricks workloads for performance, cost, and reliability (Photon, cluster policies, Liquid Clustering, Auto Loader, etc.) • Integrate Databricks with client ecosystems (Azure, AWS, GCP, Snowflake, Kafka, legacy systems, etc.) • Support client workshops, proof-of-concepts (POCs), and knowledge transfer sessions • Deliver projects following consulting methodologies while meeting quality, timeline, and budget expectations • Document architectures, runbooks, and best practices for client use • Participate in solutioning activities (scoping, estimation, technical demos) as needed

🎯 Requirements

• 3 -5 years of hands-on Databricks experience (or strong Spark experience with significant recent Databricks work) • Proven experience delivering Databricks projects in a consulting or professional services environment (preferred) or equivalent client-facing project delivery • Strong proficiency in PySpark, Spark SQL, Python, and SQL • Deep experience with Delta Lake, Unity Catalog, Delta Live Tables, and Databricks Jobs • Hands-on experience with Git version control, pull requests, code reviews, and collaborative development workflows • Cloud platform experience (Azure Databricks, AWS, or GCP - at least one) • Excellent client-facing and communication skills - able to explain complex concepts to both technical and non-technical audiences • Solid understanding of data governance, security, and Lakehouse best practices • Bachelor's degree in Computer Science, Engineering, or related field (or equivalent experience)

🏖️ Benefits

• As a Databricks Data Engineer, you will work directly with clients across multiple industries to design, implement, and optimize Databricks-based data solutions • You will be a key member of our Professional Services delivery teams, delivering high-quality projects on time and within scope while building strong client relationships • This is a client-facing role that combines hands-on technical delivery with consulting best practices • Collaborate with client data teams to ensure successful adoption and handover of solutions