Senior Data Engineer – Azure Data Lakehouse

🔥 0 minutes ago

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Minsait

WebsiteLinkedInAll Job Openings

10,000+ employees

Founded 2016

🤖 Artificial Intelligence

🔒 Cybersecurity

🏢 Enterprise

Artificial Intelligence • Cybersecurity • Enterprise

Minsait is a global leader in technology and consulting, offering a wide range of innovative solutions and services to enhance business operations and societal transformation. With a focus on human-centered technology, Minsait provides services such as AI, cybersecurity, intelligent workspace design, and cloud platform management. The company partners with various industries including public administration, healthcare, utilities, transportation, and hospitality, tailoring its solutions to meet sector-specific needs. Minsait is known for its impactful and pioneering approach, making technology more accessible and transformative.

📋 Description

• Design, develop, and maintain data ingestion and transformation pipelines in an Azure Data Lakehouse. • Build solutions using Azure Databricks with PySpark and SQL for data processing and modeling. • Implement and evolve integration and orchestration pipelines using Azure Data Factory. • Perform data audits, validations, and reconciliations using advanced SQL queries. • Monitor pipeline executions, logs, and performance, proposing improvements for performance, architecture, and cost. • Document pipelines, data flows, and architecture following corporate standards. • Work on integration across multiple data sources and environments. • Ensure best practices in data governance, quality, and security.

🎯 Requirements

• Solid experience with Azure Data Lakehouse. • Experience with Azure Databricks (PySpark and SQL). • Advanced proficiency in PySpark, Python, and SQL (Spark SQL and T-SQL). • Knowledge of Data Lakehouse architecture, Delta Lake, and Unity Catalog. • Experience with Azure Data Factory for pipeline ingestion and orchestration. • Experience with Azure Synapse Analytics. • Experience with Git, Azure DevOps, and CI/CD pipelines. • Experience with versioning pipelines in corporate environments. • Knowledge of Delta Sharing, SFTP, and SQL-based audits. • Knowledge of data governance using Unity Catalog is a plus. • Experience in projects integrating multiple Data Lakes using Delta Sharing is a plus. • Knowledge of performance and cost optimization in Azure environments is a plus. • Experience in enterprise data architecture and data engineering best practices is a plus. • Bachelor's degree in Computer Science, Computer Engineering, Information Systems, Software Engineering, Systems Analysis and Development, or a related field.

🏖️ Benefits

• Company-subsidized Health Plan for the employee. • Option to include dependents in the Health Plan with payroll-deduction. • Dental Assistance (optional). • Option to include dependents in the Dental Assistance Plan with payroll-deduction. • Meal Allowance or Food Voucher. • Transportation Voucher (optional). • Impact & Care - Personal Guidance Program offering confidential emotional support and counseling in psychological, legal, financial, social, and pet-related matters at no cost for the employee and legal dependents. • Gympass - Wellhub (Access to over 700 gyms across Brazil with plans starting at BRL 29.90, payroll-deducted). • Option to include dependents in Gympass - Wellhub (up to 3 dependents - paid via credit card). • Access to Udemy via our intranet. • Partnerships with major consumer brands. • Agreement with SESC for employee and dependents. • Discounts with educational institutions (undergraduate and postgraduate) and language/certification schools. • Group life insurance.