Data Engineer

August 5, 2023

Input Output (IOHK)

WebsiteLinkedInAll Job Openings

IOG is one of the world's pre-eminent blockchain infrastructure research and engineering companies.

Blockchain • Cyrptocurrency • Research • Development • Consultancy

201 - 500

Description

•Develop and maintain automated data ingestion pipelines from source code repositories, social media, and on-chain analytics. •Develop and maintain data cleaning and transformation solutions. •Develop web solutions (both back-end and front-end) for exploring data insights. •Maintain machine learning (ML) models in production pipelines. •Develop and maintain a knowledge management system to enable content by the systems’ customers. •Integrate and maintain Large Language Models (LLM) within the processing pipeline and develop a prompt-like user interface. •Maintain the remote infrastructure required to store, process, and visualize data.

Requirements

•Knowledge of big data processing platforms (e.g., Spark, Databricks, Google BigQuery), data orchestration (e.g., Airflow), and data manipulation libraries (e.g., Pandas, Polars). •Familiarity with machine learning platforms and libraries such as TensorFlow, PyTorch, or similar. •Knowledge of front-end languages and libraries (e.g., HTML/ CSS, JavaScript, XML, jQuery). •Knowledge of back-end languages (e.g., Python). •Knowledge of relational databases (e.g., PostgreSQL), web servers (e.g., Nginx), and UI/UX design. •Knowledge of Docker, Kubernetes, and familiarity with Continuous Integration and Continuous Delivery (CI/CD) pipelines. •Experience with cloud services (e.g., AWS, Google Cloud) •Ability to communicate well both verbally and in writing, with both technical and non-technical partners.

Benefits

•Flexible schedule •Remote work •Laptop reimbursement •New starter package to buy hardware essentials (headphones, monitor, etc) •Learning & Development opportunities •Competitive PTO and Sick Leave plan