Big Data Engineer

Job not on LinkedIn

November 20

Apply Now

Receive Emails with Similar Jobs

Weekday (YC W21)

WebsiteLinkedInAll Job Openings

Human Resources • SaaS • Recruitment

Weekday is a modern recruitment platform that combines AI technologies with a vast database of potential candidates, aiming to streamline the hiring process for companies in India. They offer various services, including a proactive outreach approach that helps employers connect with top talent, as well as tools for candidates to easily apply for jobs. Weekday's emphasis on candidate engagement through multiple channels, including email, WhatsApp, and phone calls, sets it apart in the competitive landscape of recruitment agencies.

11 - 50 employees

Founded 2021

☁️ SaaS

🎯 Recruiter

📋 Description

• Design, build, and optimize large-scale real-time streaming data pipelines. • Develop robust data ingestion and processing frameworks using Apache Spark Streaming or Hadoop streaming tools. • Use Java to implement performant, production-ready distributed systems. • Ensure low latency, high throughput, scalability, and fault tolerance across streaming workflows. • Collaborate with product, data, and engineering teams to integrate streaming pipelines into core platforms. • Diagnose, troubleshoot, and improve streaming system performance and reliability.

🎯 Requirements

• 4+ years of experience as a Big Data Engineer. • Strong command of real-time streaming technologies (Kafka, Spark Streaming, Flink, or similar). • Advanced proficiency in Java for core development. • Hands-on experience with Hadoop or Apache Spark (streaming use cases only). • Solid understanding of distributed systems and scalable architecture patterns. • Experience in monitoring, tuning, and debugging streaming pipelines. • Background working in high-scale or big-tech engineering environments.