Senior Data Engineer

March 19

FairMoney

WebsiteLinkedInAll Job Openings

Building the leading digital bank for Emerging Markets!

Finance • Technology • Lending • Loan • Mobile Loans

201 - 500

💰 Seed Round on 2018-11

Description

• Ensure data quality and availability for all data consumers, mainly data science and BI teams. ingest raw data into our BigQuery DataWarehouse • Make sure data is processed and stored efficiently: work with backend teams to offload data from backend storage • Work with data scientists to build real-time machine learning feature computation pipelines • Spread best practices in terms of data architecture across all tech teams • Effectively form relationships with the business in order to help with the adoption of data-driven decision-making. • You will be part of the Datatech team, sitting right between data producers and data consumers. You will help build the central nervous system of our real-time data processing layer by building an ecosystem around data contracts between producers and consumers.

Requirements

• Languages: Python and Scala • Big data processing frameworks: • Streaming: Apache Flink • Batch: all or one of Apache Spark - Apache Flink - Apache Beam • Streaming services: Apache Kafka / AWS Kinesis • Managed cloud services: one of AWS EMR / AWS Kinesis Data Analytics • Docker • Building REST APIs • Deployment/ management of stateful streaming jobs • The Kafka ecosystem: Kafka connects mainly • Infrastructure as code frameworks (Terraform) • Architecture around data contracts: Avro Schemas management, schema registries (Confluent Kafka / AWS Glue) • Kubernetes • Data quality frameworks (Great Expectations / Aws Deequ)

Benefits

• Training & Development • Family Leave (Maternity, Paternity) • Paid Time Off (Vacation, Sick & Public Holidays) • Remote Work