Data Engineer

Job not on LinkedIn

1 hour ago

Apply Now

Receive Emails with Similar Jobs

Weekday (YC W21)

WebsiteLinkedInAll Job Openings

Human Resources • SaaS • Recruitment

Weekday is a modern recruitment platform that combines AI technologies with a vast database of potential candidates, aiming to streamline the hiring process for companies in India. They offer various services, including a proactive outreach approach that helps employers connect with top talent, as well as tools for candidates to easily apply for jobs. Weekday's emphasis on candidate engagement through multiple channels, including email, WhatsApp, and phone calls, sets it apart in the competitive landscape of recruitment agencies.

11 - 50 employees

Founded 2021

☁️ SaaS

🎯 Recruiter

📋 Description

• **This role is for one of the Weekday's clients******Min Experience: 4 years • Location: India • JobType: full-time • We are seeking an experienced Data Engineer with strong expertise in Databricks and modern data engineering practices. The ideal candidate will have 4+ years of hands-on experience in developing scalable data pipelines, managing distributed data systems, and supporting end-to-end CI/CD processes. This role involves architecting and optimizing data workflows that enable seamless data-driven decision-making across the organization.

🎯 Requirements

• **Responsibilities** • - Design, build, and maintain scalable ETL/ELT pipelines for large-scale datasets using Spark, Hive, or Glue. • - Develop and optimize data integration workflows using ETL tools such as Informatica, Talend, or SSIS. • - Write, optimize, and maintain complex SQL queries for data transformation and analytics. • - Collaborate with cross-functional teams including data scientists, analysts, and product stakeholders to translate requirements into technical solutions. • - Deploy data workflows using CI/CD pipelines and ensure smooth automated releases. • - Monitor and optimize data workflows for performance, scalability, and reliability. • - Ensure data accuracy, governance, security, and compliance across pipelines. • - Work with cloud-based data platforms such as Azure (ADF, Synapse, Databricks) or AWS (EMR, Glue, S3, Athena). • - Maintain clear documentation of data systems, architectures, and processes. • - Provide mentorship and technical guidance to junior team members. • - Stay current with emerging data engineering tools, technologies, and best practices. • **What You’ll Bring** • - Bachelor’s degree in IT, Computer Science, or related field. • - 4+ years of experience in data engineering and distributed data processing. • - Strong hands-on experience with Databricks or equivalent technologies (Spark, EMR, Hadoop). • - Proficiency in Python or Scala. • - Experience with modern data warehouses (Snowflake, Redshift, Oracle). • - Solid understanding of distributed storage systems (HDFS, ADLS, S3) and formats such as Parquet and ORC. • - Familiarity with orchestration tools such as ADF, Airflow, or Step Functions. • - Databricks Data Engineering Professional certification (preferred / required as needed). • - Experience in multi-cloud or migration-based projects is a plus.