Data Engineer, Databricks

Job not on LinkedIn

🔥 1 minute ago

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Livefront

WebsiteLinkedInAll Job Openings

201 - 500 employees

Founded 2001

💼 Consulting

🤖 Artificial Intelligence

🤝 B2B

💰 Private equity on 2023-10

Consulting • Artificial Intelligence • B2B

Livefront is a digital product and engineering consultancy that builds AI-first products, platforms, and intelligent agents for enterprise clients. The firm combines strategy, product design, AI & automation, digital engineering, and data & analytics in a co-creative model (called MESH) to accelerate digital transformation and ship customer-facing software from discovery through scale. Livefront highlights work across healthcare, retail, automotive, and large enterprises—delivering AI companions, revenue-optimization agents, conversational/LLM integrations, platform engineering, and modern data pipelines to improve product outcomes and operational efficiency. The company positions itself as a partner-first, B2B services provider focused on measurable business impact.

📋 Description

• Design and build production data pipelines using Lakeflow Declarative Pipelines, Autoloader, and Structured Streaming, with end-to-end ownership of ingestion, transformation, data quality expectations, and CI/CD deployment via Declarative Automation Bundles. • Architect and implement Lakehouse solutions on Databricks — medallion architecture, Delta Lake, Unity Catalog — tailored to the client's analytics, AI, and application needs. • Build and maintain Databricks transformation layers — DLT pipelines, PySpark notebooks, and dbt — with data quality constraints and SLAs baked in. • Design and maintain the data and AI foundations — Unity Catalog, Feature Store, MLflow, and Model Serving — that power production ML, agent workflows, and AI-enabled digital products. • Collaborate with product and backend engineers to design data models, APIs, and application data contracts — ensuring the platform serves the product, not just the warehouse. • Consult with clients to understand their data challenges, develop data strategies, and implement sustainable solutions. • Adapt your approach based on project needs — sometimes leading data architecture discussions with clients, other times supporting internal teams with specialized data expertise. • Work within multi-cloud environments — primarily AWS and Azure — anchoring data platform recommendations around Databricks where it fits the client's architecture and goals. • Champion data governance through Unity Catalog — access control, lineage, data quality policies, and compliance — as a first-class part of every engagement, not an afterthought. • Design data-to-application architectures — including Lakebase-backed services and Databricks Apps — that connect governed data to AI workflows, digital products, and user-facing experiences. • Help build Livefront's Databricks practice — contributing to accelerators, internal enablement, certification goals, and Databricks partner go-to-market materials alongside delivery work.

🎯 Requirements

• 3-5 years of data engineering experience with at least 2 years in production Databricks environments, preferably in a consulting or client delivery context. • Solid working knowledge of AWS and Azure cloud services relevant to Databricks deployments — storage, networking, IAM, and compute — with GCP familiarity a plus. • Deep, production-grade Databricks expertise: Lakeflow Declarative Pipelines, Autoloader, Structured Streaming, Lakeflow Jobs, Unity Catalog (including fine-grained access control and lineage) — demonstrated through shipped production workloads, not prototypes. • Proven experience designing Lakehouse architectures — medallion patterns, Delta Lake table design, partitioning, Z-ordering, and query optimization — at production scale. • Hands-on experience with data pipeline testing, observability, and CI/CD for data — including unit testing, data quality frameworks, and version-controlled deployments via Git and Declarative Automation Bundles. • Strong proficiency in SQL and Python, with the ability to write clean, performant, and maintainable code. • Understanding of data modeling, schema design, and query optimization. • Excellent communication skills with the ability to explain complex data concepts to both technical and non-technical stakeholders. • Strong problem-solving skills with the ability to navigate ambiguous requirements and deliver pragmatic solutions. • Above-average discipline and personal organization skills. • Obvious comfort with critique and peer review in the context of an iterative development process. • A demonstrated hunger for personal and professional growth. • A self-evident love and care for the craft of data engineering.

🏖️ Benefits

• You want to work with passionate and talented people who are always looking for ways to make things better. • You desire a work environment where respect, mutual trust, and egoless collaboration are paramount. • You want colleagues who take their work seriously but not themselves, and who know how to let loose and have a good time. • You like being part of a team with a reputation for excellence that gives back to the community by educating, mentoring, and sponsoring. • You want to work on products and accounts that have outsized impact and reach. • You believe in sweating the details, giving a damn about quality, and taking pride in going the extra mile. • You want to help build a data practice specialization from the ground up — shaping how we go to market with Databricks, what we build as accelerators, and what it means to do this kind of work at a digital product company.