Senior Data Engineer, Databricks

Job not on LinkedIn

🔥 1 minute ago

🇧🇷 Brazil – Remote

⏰ Full Time

🟠 Senior

🚰 Data Engineer

👻 Ghost score 13%

What it means: the chance that no one is really hiring for this job.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Compass

WebsiteLinkedInAll Job Openings

10,000+ employees

🏠 Real Estate

📱 Media

Real Estate • Media

Compass is a real-estate-focused content and services site that provides detailed market analysis, buying/selling/renting guides, mortgage and financing information, and home improvement and renovation advice. The site offers resources for homebuyers, sellers, renters, agents, and real estate investors — including articles on appraisals, affordable housing, investment strategies, staging and property maintenance. Compass aims to help users make informed decisions across the housing lifecycle through timely market updates and practical how-to content.

📋 Description

• Develop, implement, and evolve Data Engineering solutions using the Databricks ecosystem • Define and implement scalable data architectures based on the Lakehouse model and Medallion Architecture (Bronze, Silver, and Gold) • Develop, optimize, and maintain data pipelines using Databricks, PySpark, Spark SQL, and Delta Lake • Implement data ingestion, processing, and integration solutions, including CDC and Structured Streaming • Orchestrate pipelines and integrate systems using Azure Data Factory (ADF) • Develop REST APIs, microservices, and automations using Python • Ensure data governance, security, lineage, and access control • Optimize performance, scalability, and processing costs • Support observability, monitoring, and production operations for data pipelines • Serve as a technical reference for the team, mentoring professionals and establishing Data Engineering best practices

🎯 Requirements

• Solid, advanced experience delivering production projects with Databricks • Advanced proficiency in PySpark, Spark SQL, Delta Lake, and Apache Spark • Experience with DLT / Lakeflow Spark Declarative Pipelines, CDC, and Structured Streaming • Experience with Liquid Clustering, Auto Loader, and Delta table optimization • Advanced knowledge of Unity Catalog, governance, security, lineage, and access control • Experience defining data architectures, including the Lakehouse model and Medallion Architecture • Experience with Azure Data Factory (ADF) for orchestration, ingestion, and data integration • Proficiency in Python for developing REST APIs, microservices, and automations • Experience with Spark performance tuning, Photon, partitioning, shuffle, and execution plan analysis • Knowledge of Databricks Workflows, CI/CD, Git, and Declarative Automation Bundles • Experience optimizing workloads, sizing compute resources, and reducing costs • Experience with observability, monitoring, and production support for data pipelines • Autonomy in resolving complex technical issues and making architectural decisions • Experience serving as a technical reference, supporting and mentoring other professionals • Required certification: Databricks Certified Data Engineer Associate • Databricks Certified Data Engineer Professional certification strongly preferred • Databricks Certified Associate Developer for Apache Spark certification is a plus • Preferred: AWS, Azure, or GCP certifications related to Data Engineering and Solution Architecture

🏖️ Benefits

• Meal and/or food allowance • Health and dental insurance • Health insurance for your pet • Home office allowance (for remote positions) • Welcome kit during onboarding • Life insurance • Zenklub, with psychological and nutritional support • TotalPass • Certification and Continuous Training Program with partners (Adobe, Anthropic, AWS, Google, Microsoft, Salesforce) • Partnerships and discounts to support your professional development • Access to UOL content • Maternity kit • Extended maternity and paternity leave • Employee referral program

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume