Data Engineer – DataBricks

Job not on LinkedIn

🕒 September 4

🇮🇳 India – Remote

⏰ Full Time

🟡 Mid-level

🟠 Senior

🚰 Data Engineer

Where this job comes from

Posted on Palo Alto Labs's careers page ↗Palo Alto Labs's careers page, not through a job board

First seen by Remote Rocketship September 4

Still open on Palo Alto Labs's site. Checked 5 hours ago.

Apply Now
Logo of Palo Alto Labs

Palo Alto Labs

51 - 200 employees

Founded 2025

💼 Consulting

🔒 Cybersecurity

🤖 Artificial Intelligence

Consulting • Cybersecurity • Artificial Intelligence

Palo Alto Labs is a technology managed-services and innovation firm that partners with businesses to deliver next-generation management technology, digital transformation, and operational excellence. The company provides managed cloud services, cybersecurity, SAP and AI capabilities, finance-as-a-service (FaaS), and data transformation and management, while designing prototypes and processes for new services and experiences. Its focus is on empowering enterprise customers and global capability centers to improve performance, security, and go-to-market readiness.

📋 Description

• Design and implement CI/CD pipelines for data pipelines and transformation projects, including dbt on Databricks, SQL, and notebooks • Orchestrate Databricks jobs and workflows end-to-end, including ingestion, transformation, and quality checks • Integrate automated testing into CI/CD, including schema and contract checks for data models and tables • Implement FinOps best practices for cost monitoring and allocation across the EDP • Automate Databricks platform operations and related services, including workspace and cluster provisioning, library and runtime management, and job deployment/configuration • Implement and maintain identity and access management for Databricks and supporting cloud resources • Manage workspace-, cluster-, table-, and view-level access controls, service principals, groups, roles, RBAC, and TBAC models • Provide Terraform modules, Airflow DAG patterns, and Databricks job templates to accelerate project onboarding • Continuously evaluate and improve tooling, pipelines, and platform architecture for reliability, security, and developer productivity • Define the code promotion process to minimize production impacts across domains • Manage end-to-end orchestration using managed Airflow • Contribute to defining and tracking platform SLA, SLO, and SLI metrics • Participate in incident response, including triage and root cause analysis

🎯 Requirements

• 4–7+ years in DevOps, Cloud Engineering, Site Reliability Engineering, or Platform Engineering • At least 2+ years supporting data/analytics platforms • Hands-on experience with Databricks in a production environment, including workspace and cluster management, jobs/workflows, and integrations with orchestration tools • Strong experience with CI/CD pipelines, such as GitHub Actions, GitLab CI, Azure DevOps, or similar • Experience with Git-based workflows • Strong experience with Infrastructure as Code (IaC) and orchestration tools for provisioning and managing Databricks and cloud infrastructure • Experience implementing automated tests and quality gates in CI/CD pipelines • Experience operating production data workloads, including monitoring, logging, performance tuning, and incident response • Scripting skills in Python, Bash, or PowerShell for automation and integration • Practical knowledge of IT infrastructure technologies, cloud computing, cybersecurity, and disaster recovery • Working knowledge of the Azure ecosystem, including designing, building, and optimizing scalable data pipelines in cloud-native environments • Ability to partner effectively with data engineers, analytics engineers, architects, and security teams • Bachelor’s degree in Computer Science, Information Technology, or related field preferred, or equivalent work experience • Experience in CPG, retail, manufacturing, or distribution environments preferred

🏖️ Benefits

• Remote work arrangement

Apply Now

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Similar Jobs

🕒 August 24

Warner Music Group

1001 - 5000

📱 Media

Forward Deployed Engineer building automation, dashboards, and AI solutions for Warner Music Group’s music businesses. Partnering with stakeholders using Python, Databricks, Snowflake, and Streamlit.

🇮🇳 India – Remote

💰 Post-IPO Debt on 2021-10

⏰ Full Time

🟡 Mid-level

🚰 Data Engineer

🕒 August 24

Shuru

51 - 200

🤖 Artificial Intelligence

🤝 B2B

🏢 Enterprise

Data Engineer building scalable data pipelines for Shuru’s AI-native engineering services. Transitioning into LLM-powered voice-agent and conversational AI solutions.

🕒 August 19

Convatec

5001 - 10000

🏥 Healthcare

💼 Consulting

🍽️ Food & Beverage

Data Engineering Lead building Microsoft Fabric and Power BI data products for Convatec’s global medical products business. Owning ingestion, modelling, reporting, governance, and data engineering team leadership.

🕒 August 13

StarTree

11 - 50

💼 Consulting

📦 Logistics

📣 Marketing

Senior Software Engineer building Apache Pinot distributed analytics systems at StarTree, a cloud software company enabling real-time data insights. Developing fault-tolerant, low-latency analytics applications at scale.

🕒 August 10

n Human Resources & Management Systems [ nHRMS ]

11 - 50

💼 Consulting

🏥 Healthcare

📦 Logistics

Senior Data Engineer / Data Architect leading Azure data architecture and analytics delivery. Designing scalable enterprise platforms, models, and solutions for technology company in India.

Built by Lior Neu-ner. I'd love to hear your feedback — Get in touch via DM or support@remoterocketship.com
Follow Remote Rocketship on LinkedInFollow Remote Rocketship on InstagramFollow Remote Rocketship on FacebookFollow Remote Rocketship on TikTok