Data Engineer

Emploi pas sur LinkedIn

🕒 il y a 10 jours

Postuler Maintenant

Trouver des Emplois à Distance Similaires

Configurer une alerte email pour les nouveaux emplois

📊 Vérifiez votre score de CV pour ce poste

Améliorez vos chances d'obtenir un entretien en vérifiant votre score de CV avant de postuler.

Télécharger le CV

Sparibis

Site WebLinkedInTous les Emplois

11 - 50 employés

Fondée en 2018

💼 Conseil

🔒 Cybersecurity

🏢 Entreprise

Consulting • Cybersecurity • Enterprise

Sparibis est une entreprise d'ingénierie et de conseil en informatique basée à Orlando. Elle fournit un support en gestion de programmes et contrôle de projets ainsi qu'en leadership SDLC et ingénierie des systèmes. Ses capacités répertoriées incluent l'ingénierie de la cybersécurité, le cloud et l'ingénierie DevSecOps, l'ingénierie de la mobilité/logiciel, les CRM et portails d'entreprise, l'ingénierie de tests et des opérations, et la gestion des données.

Description

• Plan, create, and maintain data architectures, ensuring alignment with business requirements. • Obtain data, formulate dataset processes, and store optimized data. • Identify problems and inefficiencies and apply solutions. • Determine tasks where manual participation can be eliminated with automation. • Identify and optimize data bottlenecks, leveraging automation where possible. • Create and manage data lifecycle policies (retention, backups/restore, etc). • In-depth knowledge for creating, maintaining, and managing ETL/ELT pipelines. • Create, maintain, and manage data transformations. • Maintain/update documentation. • Create, maintain, and manage data pipeline schedules. • Monitor data pipelines. • Create, maintain, and manage data quality gates (Great Expectations) to ensure high data quality. • Support AI/ML teams with optimizing feature engineering code. • Expertise in Spark/Python/Databricks, Data Lake and SQL. • Create, maintain, and manage Spark Structured Steaming jobs, including using the newer Delta Live Tables and/or DBT. • Research existing data in the data lake to determine best sources for data. • Create, manage, and maintain ksqlDB and Kafka Streams queries/code Data driven testing for data quality. • Maintain and update Python-based data processing scripts executed on AWS Lambdas. • Unit tests for all the Spark, Python data processing and Lambda codes. • Maintain PCIS Reporting Database data lake with optimizations and maintenance (performance tuning, etc). • Streamlining data processing experience including formalizing concepts of how to handle lake data, defining windows, and how window definitions impact data freshness.

🎯 Exigences

• 5+ years of IT experience focusing on enterprise data architecture and management to include data flow charts, diagrams, and other technical documentation. • Experience with Databricks, Structured Streaming, Delta Lake concepts, and Delta Live Tables required. • Python development experience required. • Experience with ETL and ELT tools such as SSIS, Pentaho, and/or Data Migration Services, and the ability to incorporate Python as required. • Advanced level SQL experience (Joins, Aggregation, Windowing functions, Common Table Expressions, RDBMS schema design, Postgres performance optimization). • Proficiency using Git for version control, including repository management, branching, merging, and pull requests. • Active CompTIA Security+ certification preferred.

🏖️ Avantages

• Applicants must be able to obtain and maintain a secret security clearance. • United States Citizenship is required as part of the eligibility criteria to be able to obtain this type of security clearance.