Data Engineer, Mid-level

Job not on LinkedIn

🔥 0 minutes ago

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Genial Investimentos

WebsiteLinkedInAll Job Openings

501 - 1000 employees

💼 Consulting

🛡️ Insurance

💸 Finance

Consulting • Insurance • Finance

Genial Investimentos is a digital investment platform that offers a wide range of financial services including fixed income investments, variable income, investment funds, private pensions, and public offers. The company aims to simplify access to the financial market, providing tools for financial education, market analysis, and various investment options for both individual and institutional clients. Genial Investimentos operates as a broker and provides advisory services, enabling customers to manage their investments effectively through a user-friendly online platform.

📋 Description

• Develop and enhance batch and streaming data pipelines using Python, Spark and Databricks • Build integrations with databases, APIs and AWS services • Model and transform data following the platform's architecture and quality standards • Monitor production pipelines, investigating failures, performance issues and root causes • Participate in code reviews and contribute to the evolution of the team's engineering standards • Work closely with business teams to understand requirements and translate needs into data solutions • Ensure best practices for data quality, security, governance and observability

🎯 Requirements

• Strong Python for data engineering, following good development practices, testing and error handling • SQL, including CTEs, window functions and query optimization • Apache Spark / PySpark, with understanding of partitioning, shuffle, joins, caching and execution plans • Delta Lake, including MERGE, OPTIMIZE, VACUUM, schema evolution and reprocessing strategies • Databricks, with experience in Jobs/Workflows and knowledge of Unity Catalog • AWS for data, primarily S3, IAM and integration/messaging services • Dimensional modeling and layered architecture, including SCD types 1 and 2 • Git and development via pull requests • Experience ingesting from relational databases and REST APIs • Nice-to-have: experience in the financial market • Nice-to-have: Structured Streaming • Nice-to-have: Databricks Asset Bundles and Terraform • Nice-to-have: CI/CD for data pipelines • Nice-to-have: Delta Lake, including MERGE, OPTIMIZE, VACUUM, schema evolution and reprocessing strategies

🏖️ Benefits

• Health Insurance • Dental Insurance • Meal Voucher • Food Voucher • Life Insurance • Total Pass • GymPass • Childcare Allowance • PPR (Performance Bonus)