DataOps Engineer

December 13, 2023

Apply Now

Get a free AI resume review

Mactores

WebsiteLinkedInAll Job Openings

Mactores is a trusted leader among businesses in providing modern data platform solutions.

Big Data • Machine Learning • Consulting • Research • Internet of Things

51 - 200

Description

• Operate the current data lake deployed on AWS with Amazon S3, Amazon EMR, and Apache Airflow • Debug and fix production issues in PySpark • Collaborate with product teams for L3/L4 production issues in PySpark • Contribute to enhance the ETL efficiency • Build CloudWatch dashboards for optimizing the operational efficiencies • Handle escalation tickets from L1 Monitoring engineers • Assign the tickets to L1 engineers based on their expertise

Requirements

• Overall 5+ years of exp in the software industry • Exp in developing architecture data applications using python or scala, Airflow, and Kafka on AWS Data platform Experience and expertise • Must have set up or led the project to enable Data Ops on AWS or any other cloud data platform • Strong data engineering experience on Cloud platform, preferably AWS • Experience with data pipelines designed for reuse and use parameterization • Experience of pipelines was designed to solve common ETL problems • Understanding or experience on various AWS services can be codified for enabling DataOps like Amazon EMR, Apache Airflow • Experience in building data pipelines using CI/CD infrastructure • Understanding of Infrastructure as code for DataOps ennoblement • Ability to work with ambiguity and create quick PoCs • Expertise in Amazon EMR, Apache Airflow, Terraform, CloudWatch is preferred • Exposure to MLOps using Amazon Sagemaker is a plus • AWS Solutions Architect Professional or Associate Level Certificate • AWS DevOps Professional Certificate

Benefits

• Fun office environment • World-class team • Equal opportunities • Competitive compensation • Health and recreational programs