Master Data Developer, Databricks

🔥 0 minutes ago

🇨🇴 Colombia – Remote

⏰ Full Time

🟡 Mid-level

🟠 Senior

🖥 Software Engineer

👻 Ghost score 10%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

CI&T

WebsiteLinkedInAll Job Openings

5001 - 10000 employees

Founded 1995

💼 Consulting

🏥 Healthcare

📣 Marketing

💰 $5.5M Venture Round on 2014-04

Consulting • Healthcare • Marketing

CI&T is a global tech transformation specialist focusing on helping organizations navigate their technology journey. With services spanning from application modernization and cloud solutions to AI-driven data analytics and customer experience, CI&T empowers businesses to accelerate their growth and maximize operational efficiency. The company emphasizes digital product design, strategy consulting, and immersive experiences, ensuring a robust support system for enterprises in various industries.

📋 Description

• Build, optimize, and maintain scalable data pipelines and ETL/ELT processes for analytics and business intelligence • Implement data architectures, including data lakes, warehouses, and integration layers • Implement data quality checks, validation processes, and monitoring systems • Develop and maintain integrations between data sources, third-party tools, and analytics platforms • Collaborate with Data & Analytics Managers and business stakeholders to understand requirements and deliver technical solutions • Monitor and optimize data pipeline performance, query efficiency, and storage costs • Create and maintain technical documentation for data architectures, pipelines, and processes • Implement security best practices and ensure compliance with data privacy regulations • Leverage AI tools and automation to enhance data engineering workflows and productivity

🎯 Requirements

• Hands-on Databricks migration with the databricks/migrate tool, including notebooks, jobs, clusters, secrets, and ACLs • Unity Catalog to Glue Catalog migration experience • Notebook and job migration, including DAG pipelines and DLT pipelines • Storage path refactoring from /mnt/ and DBFS references to S3-backed locations • Data migration from Delta Lake format to Iceberg format on S3, including Delta tables, raw files, and incremental sync • Streaming experience with Auto Loader / DLT checkpoint re-bootstrap and event_id deduplication • Validation using row counts, checksums, sample queries, and ETL runtime comparison • Recreate secret scopes in AWS Secrets Manager • Experience with AWS services: S3, Glue, Secrets Manager, IAM basics, Athena, Lake Formation, and EMR • Programming and query languages: Spark SQL, PySpark, Java, Python, and Scala • Agile pod delivery, including daily ticket updates, standups, and code review

🏖️ Benefits

• Premium Healthcare • Meal voucher • Maternity and Parental leaves • Mobile services subsidy • Sick pay-Life insurance • CI&T University • Colombian Holidays • Paid Vacations