Senior Data Engineer

Job not on LinkedIn

🔥 0 minutes ago

🇮🇳 India – Remote

⏰ Full Time

🟠 Senior

🚰 Data Engineer

👻 Ghost score 10%

What it means: the chance that no one is really hiring for this job.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.Why this score:

• Just posted.
• Jobs at this company usually fill in 35 days.
• No salary listed.

Apply Now

Find Similar Remote Jobs

Sumo Logic

WebsiteLinkedInAll Job Openings

501 - 1000 employees

Founded 2010

☁️ SaaS

🔒 Cybersecurity

💰 $110M Series G on 2019-05

SaaS • Cybersecurity • Cloud Computing

Sumo Logic is a cloud-based machine data analytics company that offers a comprehensive platform for monitoring, troubleshooting, automating, and defending IT infrastructures. The company specializes in cloud security intelligence, leveraging AI and machine learning to provide enhanced observability and security threat detection. It is known for its powerful log management capabilities, which help organizations unlock cloud security and efficiently troubleshoot system issues. Sumo Logic's platform integrates seamlessly with cloud services, providing infrastructure monitoring and application observability to modernize IT operations. Their solutions are tailored to various use cases, including DevSecOps, cloud migration, and digital customer experience enhancement.

📋 Description

• Design, develop, and optimize scalable data pipelines using Databricks, Apache Spark, PySpark, Python, SQL, and Delta Lake • Design and implement Bronze, Silver, and Gold medallion architecture layers • Build and maintain integrations with enterprise applications, REST APIs, databases, and other source systems • Develop reusable ingestion frameworks for onboarding new data sources • Build batch and near-real-time ingestion pipelines with incremental loads, CDC, schema evolution, and error recovery • Configure and manage Databricks Workflows and implement pipelines using Auto Loader and Lakeflow Spark Declarative Pipelines • Optimize Spark and Delta Lake workloads, query performance, partitioning, joins, data-skew handling, and compute costs • Implement Unity Catalog governance, access controls, lineage, auditing, masking, and row-level security • Maintain data workflows on Amazon Redshift and Amazon EC2 • Collaborate with Engineering and IT/Cloud teams on connectivity, IAM permissions, S3 access, deployments, and operational issues • Follow version control, code review, automated testing, CI/CD, and technical documentation practices • Partner with Data Analytics and business stakeholders to deliver trusted, documented, analytics-ready datasets • Provide technical guidance and improve automation, engineering standards, platform reliability, and resource utilization • Report directly to the Senior Manager - Business Insights & Analytics and collaborate with Data Analytics, Engineering, IT/Cloud, Security, and business stakeholders

🎯 Requirements

• 5+ years of experience in data engineering or a related technical role, including developing and supporting production data pipelines • Strong hands-on expertise in Databricks, including developing, troubleshooting, and optimizing production workloads • Strong proficiency in Python/PySpark, SQL, and Apache Spark for scalable data processing and transformation • Practical experience with Delta Lake, including table management, incremental processing, and performance optimization • Experience with data architecture and modeling, including medallion architecture and the development of analytics-ready datasets • Experience with Databricks Workflows or equivalent orchestration solutions • Proven experience building data ingestion pipelines and integrating REST APIs, databases, and enterprise applications • Understanding of reliable data processing patterns, including retries, idempotency, schema evolution, backfills, and failure recovery • Working knowledge of AWS services and cloud infrastructure, particularly IAM roles and policies, S3 storage and access management, and secure connectivity between source systems and data platforms • Experience with Git/GitHub, code reviews, automated testing, and CI/CD practices • Strong problem-solving skills and the ability to independently design, build, deploy, and maintain production data solutions • Good communication and collaboration skills to work effectively with Data Analytics, Engineering, IT/Cloud, Security, and business stakeholders • Experience with Databricks Lakeflow Spark Declarative Pipelines, Auto Loader, and advanced incremental ingestion patterns is desirable • Experience implementing Unity Catalog governance is desirable • Experience with Databricks Genie, Genie Agents, and Genie Code is desirable • Experience in AI/ML, Data Science, LLMs, contextualization, and Amazon Bedrock is desirable

🏖️ Benefits

• Remote work arrangement • AI-powered Talent Matching tool used in recruitment • Privacy and data protection compliance information provided

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume