Data Engineer

🔥 3 hours ago

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

GFT Technologies

WebsiteLinkedInAll Job Openings

10,000+ employees

Founded 1987

🔒 Cybersecurity

📋 Compliance

☁️ SaaS

Cybersecurity • Compliance • SaaS

GFT Technologies is a leading technology and digital transformation company that specializes in providing advanced solutions for consent management and data privacy compliance. Their flagship product, Cookiebot, enables businesses to automate user consent collection efficiently, ensuring adherence to complex privacy regulations such as GDPR and CCPA. GFT's solutions facilitate seamless integration into existing digital infrastructures, helping over 600,000 customers globally manage user data transparently and securely.

📋 Description

• Rebuild legacy Data Warehouse pipelines in Databricks using PySpark and Spark SQL; • Implement bronze, silver and gold layers following the medallion architecture patterns; • Apply CDC and batch ingestion patterns according to project guidelines; • Execute reconstruction waves by domain, running alongside the legacy DW until cutover; • Implement business rules and transformations with automated tests; • Perform reconciliation and parity validation between legacy environment data and the Lakehouse; • Optimize pipeline performance and costs through partitioning, OPTIMIZE, Z-ORDER and workload sizing; • Contribute to technical documentation for migrated rules; • Support the prioritization and execution of migration waves;

🎯 Requirements

• Advanced knowledge of PySpark and Python; • Experience developing large-scale batch pipelines; • Advanced SQL skills; • Experience with relational and dimensional data modeling; • Production experience with Databricks and Delta Lake; • Knowledge of the medallion architecture (bronze, silver and gold); • Experience with Delta Live Tables, Lakeflow and Asset Bundles; • Experience migrating or rebuilding ETL pipelines; • Experience translating business rules from legacy platforms to Spark; • Knowledge of CDC and batch ingestion from relational databases; • Experience with AWS (S3, Glue, EMR, Athena, Lambda, DMS and Step Functions); • Knowledge of Azure Synapse (SQL Pools and Pipelines); • Experience with data quality and reconciliation; • Knowledge of automated testing and Data Quality controls; • Experience with Git and CI/CD applied to data engineering;

🏖️ Benefits

• Multi-benefits card – you choose how and where to use it. • Study scholarships for Undergraduate, Graduate, MBA and Language courses. • Certification incentive programs. • Flexible working hours. • Competitive salaries. • Annual performance review with a structured career plan. • Opportunity for international career development. • Wellhub and TotalPass. • Private pension plan. • Childcare assistance. • Medical insurance. • Dental insurance. • Life insurance.