Senior Data Engineer

Job not on LinkedIn

🕒 3 days ago

Apply Now

Find Similar Remote Jobs

Receive Emails For Remote Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

FCamara Consulting & Training

WebsiteLinkedInAll Job Openings

1001 - 5000 employees

🛍️ eCommerce

🤖 Artificial Intelligence

Consulting • eCommerce • Artificial Intelligence

FCamara Consulting & Training is a Brazilian multinational firm specializing in digital transformation and consulting services. With over 17 years in the market, FCamara's ecosystem focuses on innovation, e-commerce, digital strategy, and the integration of technology solutions such as artificial intelligence and cybersecurity. The company collaborates with a variety of sectors including retail, healthcare, finance, and logistics, empowering more than 1,000 businesses to evolve in the digital realm.

📋 Description

• Data Integration and Collection: Aggregates structured, semi-structured and unstructured data from multiple sources (databases, APIs, files); • Real-time and Batch Data Ingestion: Designs scalable ingestion architectures using Pub/Sub for streaming and Cloud Storage/Dataflow for batch processing, ensuring data integrity in BigQuery; • Data Processing and Transformation: Builds pipelines to transform raw data with standardization, deduplication, filtering and enrichment processes; • Governance and Operations: Manages orchestration via Cloud Composer with a focus on idempotency and resilience, implements proactive monitoring (logs/alerts) and technical documentation to ensure continuity and maintainability of solutions; • Performance Optimization: Leads FinOps initiatives, optimizing BigQuery slot utilization, storage costs and job efficiency in Dataflow/Dataproc to reduce TCO (Total Cost of Ownership);

🎯 Requirements

• Bachelor's degree in IT-related fields such as Information Systems, Computer Science, or Computer Engineering. • 5 to 8 years of experience in Data Engineering or similar roles; • Advanced proficiency in SQL and Python (focused on large-volume data handling and automation); experience with Java is a plus for custom Dataflow pipelines; • Strong experience with modern ETL/ELT architectures; proficiency with Cloud Composer (Airflow) for orchestration and Dataflow (Apache Beam) or dbt for transformations; • Expertise in the GCP ecosystem: BigQuery (partitioning/clustering), Dataproc, Cloud Functions and IAM for data security; • Experience with Data Lake and Data Warehouse architectures, including data modeling and pipeline optimization; • Proficiency in Terraform (IaC) and Azure Pipelines (CI/CD) for version control in Azure Repos and artifact management; • Proven ability in monitoring and troubleshooting complex processes in production environments.

🏖️ Benefits

• Not specified