Senior Data Engineer – Databricks

Job not on LinkedIn

🔥 0 minutes ago

🇧🇷 Brazil – Remote

⏰ Full Time

🟠 Senior

🚰 Data Engineer

👻 Ghost score 10%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

INDT - Instituto de Desenvolvimento Tecnológico

WebsiteLinkedInAll Job Openings

201 - 500 employees

Founded 2001

💼 Consulting

🏭 Manufacturing

🏥 Healthcare

Consulting • Manufacturing • Healthcare

INDT - Instituto de Desenvolvimento Tecnológico is a Brazilian research, development and innovation (PD&I) center founded in Manaus that provides multidisciplinary technological solutions and services to industry. It offers advanced materials development, biotechnology integration for sustainability and supply-chain traceability, communication and network engineering including WiFi-Mesh and private 5G, advanced manufacturing and automation, cybersecurity (SOC) for industrial environments, laboratory testing and prototyping, and professional training in Industry 4. 0 technologies. The institute partners with industry and public programs to deliver R&D, testing, certification and applied training.

📋 Description

• Implement and enhance the Databricks data platform • Develop and enhance data pipelines using Spark/PySpark • Build ingestion, transformation, and processing solutions for large volumes of data • Work with Delta Lake and resources across the Databricks ecosystem • Contribute to the architecture and modeling of scalable data solutions • Implement data engineering best practices related to quality, governance, and observability • Identify and resolve performance issues in data pipelines and processes • Optimize processing, storage, and queries • Work with Unity Catalog and related data governance and security capabilities • Participate in defining and implementing pipeline orchestration and integration strategies • Collaborate with technology and business teams to define the best technical solutions • Ensure the quality, reliability, scalability, and maintainability of developed solutions • Contribute to the advancement of the project's data engineering standards and best practices

🎯 Requirements

• Solid, hands-on experience with Databricks • Advanced experience with Apache Spark and PySpark • Experience with Delta Lake • Knowledge of and experience with Unity Catalog • Experience implementing and enhancing data pipelines • Knowledge of data architecture and data engineering best practices • Experience optimizing and troubleshooting performance in Spark/Databricks • Experience processing large volumes of data • Ability to work independently when analyzing problems and defining technical solutions • Experience in corporate/enterprise environments • Preferred: Databricks certification • Preferred: Experience with AWS • Preferred: Experience with Terraform and Infrastructure as Code (IaC) practices • Preferred: Knowledge of or experience with Kafka and other streaming technologies • Preferred: Experience with data quality • Preferred: Knowledge of observability tools and practices • Preferred: Previous experience on large-scale projects • Preferred: Experience working in high-volume environments with high-availability requirements

🏖️ Benefits

• Collaborative environment • Learning and career growth opportunities • Excellent workplace • Inspiring colleagues and leaders • Freedom to propose and develop innovative projects • Autonomy and ownership • Opportunities to develop your talents • An environment where you feel valued