Data Engineer – DataBricks

Job not on LinkedIn

🕒 2 days ago

🇮🇳 India – Remote

⏰ Full Time

🟡 Mid-level

🟠 Senior

🚰 Data Engineer

👻 Ghost score 15%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Palo Alto Labs

WebsiteLinkedInAll Job Openings

51 - 200 employees

Founded 2025

💼 Consulting

🔒 Cybersecurity

🤖 Artificial Intelligence

Consulting • Cybersecurity • Artificial Intelligence

Palo Alto Labs is a technology managed-services and innovation firm that partners with businesses to deliver next-generation management technology, digital transformation, and operational excellence. The company provides managed cloud services, cybersecurity, SAP and AI capabilities, finance-as-a-service (FaaS), and data transformation and management, while designing prototypes and processes for new services and experiences. Its focus is on empowering enterprise customers and global capability centers to improve performance, security, and go-to-market readiness.

📋 Description

• Design and implement CI/CD pipelines for data pipelines and transformation projects, including dbt on Databricks, SQL, and notebooks • Orchestrate Databricks jobs and workflows end-to-end, including ingestion, transformation, and quality checks • Integrate automated testing into CI/CD, including schema and contract checks for data models and tables • Implement FinOps best practices for cost monitoring and allocation across the EDP • Automate Databricks platform operations and related services, including workspace and cluster provisioning, library and runtime management, and job deployment/configuration • Implement and maintain identity and access management for Databricks and supporting cloud resources • Manage workspace-, cluster-, table-, and view-level access controls, service principals, groups, roles, RBAC, and TBAC models • Provide Terraform modules, Airflow DAG patterns, and Databricks job templates to accelerate project onboarding • Continuously evaluate and improve tooling, pipelines, and platform architecture for reliability, security, and developer productivity • Define the code promotion process to minimize production impacts across domains • Manage end-to-end orchestration using managed Airflow • Contribute to defining and tracking platform SLA, SLO, and SLI metrics • Participate in incident response, including triage and root cause analysis

🎯 Requirements

• 4–7+ years in DevOps, Cloud Engineering, Site Reliability Engineering, or Platform Engineering • At least 2+ years supporting data/analytics platforms • Hands-on experience with Databricks in a production environment, including workspace and cluster management, jobs/workflows, and integrations with orchestration tools • Strong experience with CI/CD pipelines, such as GitHub Actions, GitLab CI, Azure DevOps, or similar • Experience with Git-based workflows • Strong experience with Infrastructure as Code (IaC) and orchestration tools for provisioning and managing Databricks and cloud infrastructure • Experience implementing automated tests and quality gates in CI/CD pipelines • Experience operating production data workloads, including monitoring, logging, performance tuning, and incident response • Scripting skills in Python, Bash, or PowerShell for automation and integration • Practical knowledge of IT infrastructure technologies, cloud computing, cybersecurity, and disaster recovery • Working knowledge of the Azure ecosystem, including designing, building, and optimizing scalable data pipelines in cloud-native environments • Ability to partner effectively with data engineers, analytics engineers, architects, and security teams • Bachelor’s degree in Computer Science, Information Technology, or related field preferred, or equivalent work experience • Experience in CPG, retail, manufacturing, or distribution environments preferred

🏖️ Benefits

• Remote work arrangement