Data Architect, Databricks

🔥 13 minutes ago

🇧🇷 Brazil – Remote

⏰ Full Time

🟡 Mid-level

🟠 Senior

🚰 Data Engineer

👻 Ghost score 10%

What it means: the chance that no one is really hiring for this job.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

CI&T

WebsiteLinkedInAll Job Openings

5001 - 10000 employees

Founded 1995

💼 Consulting

🏥 Healthcare

📣 Marketing

💰 $5.5M Venture Round on 2014-04

Consulting • Healthcare • Marketing

CI&T is a global tech transformation specialist focusing on helping organizations navigate their technology journey. With services spanning from application modernization and cloud solutions to AI-driven data analytics and customer experience, CI&T empowers businesses to accelerate their growth and maximize operational efficiency. The company emphasizes digital product design, strategy consulting, and immersive experiences, ensuring a robust support system for enterprises in various industries.

📋 Description

• Build, optimize, and maintain scalable data pipelines and ETL/ELT processes • Implement data architectures including data lakes, warehouses, and integration layers • Implement data quality checks, validation processes, and monitoring systems • Develop and maintain integrations between data sources, third-party tools, and analytics platforms • Collaborate with Data & Analytics Managers and business stakeholders to understand requirements and deliver technical solutions • Monitor and optimize pipeline performance, query efficiency, and storage costs • Create and maintain technical documentation for data architectures, pipelines, and processes • Implement security best practices and ensure compliance with data privacy regulations • Leverage AI tools and automation to enhance data engineering workflows and productivity

🎯 Requirements

• Hands-on Databricks migration with the databricks/migrate tool, including notebooks, jobs, clusters, secrets, and ACLs • Unity Catalog to Glue Catalog migration experience • Notebook and job migration, including notebooks, DAG pipelines, and DLT pipelines • Storage path refactoring from /mnt/ and DBFS references to S3-backed locations • Data migration from Delta Lake format to Iceberg format on S3, including Delta tables, raw files, and incremental sync • Streaming experience with Auto Loader and DLT checkpoint re-bootstrap • event_id deduplication • Validation using row counts, checksums, sample queries, and ETL runtime comparison • Recreating secret scopes in AWS Secrets Manager • Experience with AWS S3, Glue, Secrets Manager, IAM basics, Athena, Lake Formation, and EMR • Proficiency in Spark SQL, PySpark, Java, Python, and Scala • Agile pod delivery, including daily ticket updates, standups, and code review • Databricks Certified Consultant certification (Associate/Professional) preferred • DBR to EMR on AWS migration experience preferred

🏖️ Benefits

• Health and dental insurance • Meal and food allowance • Childcare assistance • Extended paternity leave • Partnerships with gyms and health and wellness professionals via Wellhub (Gympass) TotalPass • Profit Sharing and Results Participation (PLR) • Life insurance • Continuous learning platform (CI&T University) • Discount club • Free online platform dedicated to physical, mental, and overall well-being • Pregnancy and responsible parenting course • Partnerships with online learning platforms • Language learning platform • Health and Well-being team, inclusion specialists, and affinity groups • Support and accommodations for people with disabilities

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume