Data Engineer

🔥 0 minutes ago

🇮🇳 India – Remote

⏰ Full Time

🟢 Junior

🟡 Mid-level

🚰 Data Engineer

🚫👨‍🎓 No degree required

👻 Ghost score 25%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

In All Media

WebsiteLinkedInAll Job Openings

1001 - 5000 employees

💼 Consulting

📣 Marketing

☁️ SaaS

Consulting • Marketing • SaaS

In All Media is a global community-driven service organization specializing in agile development services. The company focuses on developing digital solutions through a collaborative approach that leverages swarming intelligence and team dynamics. They are dedicated to aligning industry needs with the interests of their technical community members, who possess extensive experience in various domains such as UX/UI design, quality engineering, cloud computing, big data, Salesforce, and more. In All Media emphasizes a secure development environment and a strong collaborative spirit, striving for innovation and excellence in software production. Their model relies on pre-established teams with substantial past experience working together, ensuring efficient and effective delivery of customized development solutions.

📋 Description

• Execute the end-to-end migration of approximately 1,000 datasets to Snowflake • Upgrade orchestration workflows from Apache Airflow 2 to Airflow 3 • Migrate legacy Hive data structures and datasets to Apache Iceberg using the Snowpark Connect framework/connectors • Reconfigure, update, and migrate Confluent/Apache Kafka streaming connections across cloud regions • Adapt existing data pipelines to target Snowflake environments • Optimize SQL queries using CTEs, window functions, and complex joins • Manage, test, and debug DAGs and workflow scheduling in Apache Airflow • Implement testing strategies and conduct hands-on code debugging to ensure data quality and consistency • Modernize streaming infrastructure across cloud regions and uphold data quality and performance standards • Collaborate with the Data Platform team on a major data platform modernization initiative

🎯 Requirements

• Minimum 2+ years of hands-on data engineering experience working with large-scale datasets, Apache Airflow, and Apache Spark • Strong software engineering fundamentals • Hands-on Python development experience • Advanced SQL skills, including complex joins, CTEs, window functions, and query tuning • Proven experience with Snowflake data modeling and schema architecture • Direct experience building and testing pipelines using Apache Airflow, Apache Kafka, and Apache Spark / PySpark • Expertise with Apache Iceberg and Apache Parquet data formats • Experience with AWS S3, including data lakes, bucket partitioning, and lifecycle policies • Ability to define testing strategies and independently debug complex distributed data pipelines • Strong communication skills and ability to work independently in a fast-paced environment • Ability to demonstrate good connectivity to an AWS WorkSpace region by completing the required connectivity test • Required English for interviews, documentation, and daily communication • Ability to overlap until at least 12:00 PM Pacific Time

🏖️ Benefits

• Fully remote work arrangement • Full-time, permanent employment • Client-provisioned AWS WorkSpace