Senior Data Engineer – Databricks

🕒 August 6

🇺🇸 United States – Remote

💵 $155k - $185k / year

⏰ Full Time

🟠 Senior

🚰 Data Engineer

🦅 H1B Visa Sponsor

This company has sponsored H1B visas in the past.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

SugarCRM

WebsiteLinkedInAll Job Openings

501 - 1000 employees

Founded 2004

💼 Consulting

📣 Marketing

☁️ SaaS

💰 Private Equity Round on 2018-08

Consulting • Marketing • SaaS

SugarCRM is a leading provider of customer relationship management (CRM) software that caters to B2B manufacturers and other industries. The company offers solutions designed to streamline complex sales cycles and improve customer service through advanced automation and AI-powered insights. SugarCRM provides a unified view of customer data, helping businesses enhance their decision-making processes and predict sales performance. Their platform supports integrations with modern and legacy systems, accommodating various business needs. SugarCRM also delivers services in marketing automation and revenue intelligence, making it a comprehensive CRM choice for businesses seeking to improve productivity and customer engagement.

📋 Description

• Own Databricks production support for the Sugar Predict data platform, including monitoring, alerting, and incident response across production data flows • Maintain and report SLA performance metrics for data pipeline delivery • Identify and implement pipeline optimizations to reduce Databricks compute costs, improve throughput, and reduce processing windows • Migrate legacy ETL/ELT pipelines to Databricks and build automation tooling for uninterrupted data delivery • Support new customer onboarding by provisioning, validating, and hardening tenant data pipelines • Design and build Databricks pipelines ingesting, transforming, and serving ERP and CRM data across Azure and AWS • Own Delta Lake architecture, including schema design, partitioning, data quality enforcement, and incremental processing • Enforce data security best practices, including role-based access control, secrets management, and compliance requirements • Implement data quality monitoring and observability for pipelines and ML model inputs • Apply multi-tenant data isolation patterns for secure, reliable customer data delivery • Partner with Enterprise Architecture to integrate pipelines with the SugarAI product ecosystem • Participate in on-call rotation and after-hours incident response across multiple time zones • Maintain technical documentation, runbooks, and architectural decision records • Apply CI/CD practices including version control, automated testing, and deployment tooling

🎯 Requirements

• 4+ years of data engineering experience • At least 2 years on Databricks or the Apache Spark ecosystem across Azure and/or AWS • Proficiency in PySpark, SQL, and Python • Track record building and operating production-grade pipelines under SLA constraints • Hands-on experience with Delta Lake, including schema evolution, ACID transactions, optimize/vacuum lifecycle, and incremental and streaming processing • Experience with pipeline performance tuning and compute optimization in production Databricks environments • Working knowledge of PostgreSQL, including query optimization, schema design, and production pipeline source/sink use • Experience supporting and maintaining legacy ETL tooling such as SSIS, Informatica, or custom Python/SQL pipelines • Experience supporting large-scale multi-tenant architectures, tenant isolation, per-tenant performance, and data privacy • Ability to collaborate across data science, product, and infrastructure teams and own end-to-end delivery • Understanding of data governance, security, and compliance principles • Preferred: Databricks workspaces across Azure and AWS, Serverless optimization, Microsoft SQL Server, ML feature engineering or feature stores, customer onboarding automation or IaC, and Databricks Certified Data Engineer certification

🏖️ Benefits

• Excellent healthcare package for you and your family • 401(k) match • Unlimited Paid Time Off • Paid Parental Leave • Online Legal Services (Rocket Lawyer) • Financial Planning Services (Origin) • Discounted Pet Insurance (Embrace Pet Insurance) • Corporate Benefit Program (Working Advantage), offering exclusive travel and entertainment offers and special discounts • Health and Wellness Reimbursement Program • Travel Discounts • Educational Resources - Career & Personal Development Program • Employee Referral Bonus Program • Opportunities to learn, excel and grow your career