Senior Data Engineer

Job not on LinkedIn

🔥 0 minutes ago

🇺🇸 United States – Remote

⏰ Full Time

🟠 Senior

🚰 Data Engineer

👻 Ghost score 12%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

AgencyBloc

WebsiteLinkedInAll Job Openings

51 - 200 employees

🛡️ Insurance

🏥 Healthcare

💰 Private Equity Round on 2021-10

Insurance • Healthcare

AgencyBloc is the #1 Recommended Insurance Industry Growth Platform designed specifically for the health, benefits, and senior insurance sectors. Since its establishment in 2008, AgencyBloc has provided robust, insurance-specific solutions including an industry-tailored CRM, automated workflow management, policy management, and commissions processing. With over 6,500 customers, AgencyBloc helps organizations accelerate their growth through advanced sales enablement and efficient customer management features.

📋 Description

• Design, build, and maintain scalable batch and incremental data pipelines on Databricks and AWS • Implement and extend Medallion bronze, silver, and gold architecture • Build and optimize dimensional and other data models for BI, reporting, and AI use cases • Develop ingestion for SaaS APIs, CRM and support systems, and relational/OLTP databases • Handle schema evolution, incremental loads, and late-arriving or malformed data safely • Write clean, tested, and documented Python/PySpark and SQL • Package reusable frameworks and utilities • Own assigned data initiatives end-to-end: design, build, test, deploy, operate, and iterate • Improve platform standards, tooling, and reusable patterns with the Data Architect • Implement data quality validation, reconciliation, and remediation • Apply cataloging, lineage, classification, documentation, and ownership metadata • Implement secure data handling, including access controls, encryption, masking, and PII handling • Apply data retention, archival, and lifecycle rules • Instrument pipelines with monitoring, alerting, logging, and SLAs/SLOs • Triage and resolve pipeline failures and data incidents; perform root-cause analysis • Optimize Spark workloads, right-size compute, and monitor pipeline costs • Mentor Data Engineers and Data Developers through code review, pairing, and technical guidance • Serve as a technical escalation point for complex pipeline, modeling, and performance problems • Participate in architecture reviews • Partner with analytics, product, and engineering stakeholders to translate data needs into trustworthy datasets • Build datasets, features, and serving structures for BI dashboards and AI/ML workloads

🎯 Requirements

• Bachelor's degree in Computer Science or equivalent experience preferred • 7+ years of experience in data engineering or analytics engineering, with at least 2 years operating at a senior level • Strong hands-on production experience with Databricks and/or Spark, including PySpark development, Delta Lake, and workload tuning • Unity Catalog experience is a plus • Experience building pipelines on AWS, including services such as S3, IAM, RDS, Glue, or comparable services • Dual-cloud exposure to Azure is a plus • Experience implementing Medallion (bronze/silver/gold) or comparable layered lakehouse architectures • Strong data modeling skills for analytical use cases; dimensional modeling required • Exposure to Data Vault or similar history-preserving patterns is a plus • Experience extracting and modeling data from relational/OLTP source systems such as MySQL, including CDC or incremental load patterns and handling imperfect source data • Expert-level SQL and strong Python • Software engineering fundamentals: version control, testing, code review, and CI/CD for data pipelines, such as GitHub Actions • Experience implementing data quality checks, pipeline observability, and monitoring/alerting in production • Working knowledge of data security practices, including access management, encryption, masking, and PII handling in regulated environments such as SOC 2 • Experience with orchestration tools such as Databricks Workflows, Airflow, or comparable tools • Strong communication skills, including ability to explain technical decisions and data issues to non-technical stakeholders • Experience working in insurance, InsurTech, or other regulated industries is a plus • Experience supporting AI/ML data needs, including feature datasets and training data preparation, is a plus • Applicants for employment in the US must have work authorization that does not now or in the future require sponsorship of a visa for employment authorization in the United States