Data Engineering Lead

🕒 April 7

Apply Now

Find Similar Remote Jobs

Receive Emails For Remote Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

MediaRadar, Inc.

WebsiteLinkedInAll Job Openings

201 - 500 employees

Founded 2007

☁️ SaaS

Advertising • SaaS

MediaRadar, Inc. is an award-winning advertising intelligence platform designed to support media sales and advertising planning. With over 4 million brands and comprehensive contact information, MediaRadar is a go-to solution for media sellers and buyers, providing insights into advertising data across various formats. The platform is widely used in the media and ad tech industry for prospecting, media buying, planning, and sales enablement. MediaRadar leverages AI-powered solutions to deliver actionable advertising intelligence, helping users make informed decisions for better media mix and revenue growth.

📋 Description

• Architect and implement complex, end-to-end data pipelines using Azure Databricks and PySpark. • Optimize Apache Spark jobs, tune Databricks units, and define cluster policies to minimize compute costs. • Develop a proactive monitoring and alerts framework to ensure 99.9% reliability. • Architect and design high-performance schemas in PostgreSQL, managing indexing, partitioning, and optimizing complex analytical queries. • Lead a lean team toward cross-trained agility, managing sprint cycles and conducting code reviews. • Anticipate future data needs and design High-Velocity Architecture. • Collaborate with ML teams to integrate automated model orchestration into robust ETL pipelines. • Facilitate seamless knowledge transfer and operational continuity across time zones.

🎯 Requirements

• Core: Python, PostgreSQL + pgvector. • Big Data: Azure Databricks, PySpark, Delta Lake • DevOps: Docker, Git, Azure DevOps, CI/CD • Qualifications • 10+ years of experience in Data or Software Engineering with deep codebase involvement. • 3+ years as a Technical Lead managing agile teams. • Proven ability to lead lean, high-impact teams while maintaining high individual output. • Experience with cross-training advocacy and scaling data processing through automation. • Desired Qualifications • Workflow Orchestration: Experience with Apache Airflow. • Containerization: Familiarity with Azure Kubernetes Service (AKS).