Data Engineer – Mid-level

Job not on LinkedIn

November 27

Apply Now

Receive Emails with Similar Jobs

Join Creative Tech

WebsiteLinkedInAll Job Openings

Software • SaaS • Compliance

Join Creative Tech is a company specializing in the conception and development of software solutions, focusing on quality and outsourcing services. They create user-centered solutions that offer innovative value while ensuring a strong return on investment through agile methodologies and rigorous testing processes. Additionally, Join Creative Tech provides qualified IT professionals to enhance service quality and productivity for their clients.

51 - 200 employees

Founded 2010

☁️ SaaS

📋 Compliance

📋 Description

• Build, test, and maintain ingestion and transformation pipelines (ETL/ELT); • Use Databricks with a focus on PySpark and SQL; • Ensure scalability and robustness for the BI team; • Optimize pipeline execution; • Perform query tuning; • Tune clusters and Delta tables to maximize performance and reduce costs; • Apply security and quality policies to pipelines; • Implement governance across Bronze → Silver → Gold layers; • Ensure quality and traceability of transformations; • Create connections and ingest data from diverse systems; • Ensure integrations are reliable and performant; • Implement automated data quality rules; • Develop gates in Databricks / Delta Live Tables (DLT); • Ensure only qualified data reaches the consumption layer; • Model structures in the Lakehouse; • Document and maintain metadata repositories and models; • Support administration of the Databricks workspace; • Manage access and enforce information security best practices.

🎯 Requirements

• Bachelor's degree in IT or any field with a postgraduate degree in IT; • Minimum of 4 years' experience in Software Development, Data Engineering and/or Data Administration; • Hands-on experience with Databricks (clusters, notebooks, Delta Lake, DLT); • Knowledge of Big Data and distributed processing; • Strong knowledge of PySpark; • Proficiency in SQL; • Experience with version control tools (Git); • Logical and physical modeling in a Lakehouse environment; • Documentation and maintenance of metadata; • Experience ingesting data from RDBMS, APIs, and Cloud Storage; • Application of security rules and governance policies; • Implementation of Data Quality Gates; • Access management in the Databricks Workspace.

🏖️ Benefits

• Infrastructure allowance; • Flexible working hours; • Birthday day off — with a surprise! • Support for training and certifications; • Access to Alura; • Partnership with FIAP; • Referral bonus; • Health insurance; • Dental plan; • Vittude — mental health platform; • Wellhub — for physical health; • New Value — discount vouchers; • SESC; • Meal and/or food allowance.