Senior Data Engineer

Job not on LinkedIn

🔥 12 hours ago

🇺🇸 United States – Remote

⏰ Full Time

🟠 Senior

🚰 Data Engineer

👻 Ghost score 10%

What it means: the chance that no one is really hiring for this job.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Penn Foster

WebsiteLinkedInAll Job Openings

201 - 500 employees

📚 Education

👥 B2C

🤝 B2B

Education • B2C • B2B

Penn Foster is an online, accredited career school and college that provides self-paced, career-focused diplomas, certificates, associate and bachelor’s degrees across vocational and professional fields. Its programs cover healthcare, veterinary technology (including an AVMA-CVTEA accredited Vet Tech program), skilled trades (HVACR, electrician, plumbing), business, IT, criminal justice, design, and high school diplomas accredited through the Distance Education Accrediting Commission. Penn Foster serves primarily individual learners with flexible, online study options and also offers training solutions for organizations and employers.

📋 Description

• Architect, build, and evolve Penn Foster Group's modern cloud data platform and AI strategy • Provide technical leadership through architecture guidance, code reviews, mentoring, and engineering best practices • Mentor and develop junior and mid-level Data Engineers • Establish standards for software development, testing, CI/CD, documentation, observability, and operational excellence • Design, build, and support scalable enterprise data pipelines using SQL, Python, Apache Spark, and Databricks • Define Databricks development practices covering Workflows, Repos, Jobs, notebooks, Python libraries, Git integration, cluster policies, SQL Warehouses, and deployment automation • Design and optimize Delta Lake architectures using Medallion patterns, Delta optimization, Liquid Clustering, and Photon • Build reusable ingestion frameworks for batch, streaming, CDC, and API-based integrations • Optimize Spark workloads for performance, scalability, reliability, and cloud cost efficiency • Develop trusted semantic data products, Genie Spaces, and Genie Ontologies for Databricks Genie • Establish practices for semantic modeling, governed metrics, business metadata, and AI-ready datasets • Collaborate with Data Science and MLOps teams on machine learning, generative AI, and advanced analytics initiatives • Design scalable Lakehouse architectures for analytics, reporting, AI, and operational data products • Implement governance with Unity Catalog, including lineage, fine-grained security, metadata management, and access controls • Champion automated testing, monitoring, observability, data quality, and production reliability • Serve as the technical escalation point for complex production issues and root cause analysis • Drive platform modernization and contribute to technical roadmaps and platform strategy • Collaborate with Business Intelligence, Product, Platform Engineering, Security, Data Governance, and MLOps teams • Translate business requirements into technical solutions supporting reporting, analytics, AI, and strategic decision-making

🎯 Requirements

• Bachelor's degree in Computer Science, Engineering, Information Systems, or a related field, or equivalent practical experience • 6+ years of experience in Data Engineering, Analytics Engineering, or Data Platform Engineering • 2+ years of experience leading technical initiatives and mentoring engineering teams • Demonstrated success developing engineers while driving engineering excellence and technical standards • Strong communication and collaboration skills with the ability to influence technical and business stakeholders • Expert knowledge of the Databricks Lakehouse Platform, including Apache Spark (PySpark), Delta Lake, Unity Catalog, Databricks Workflows & Jobs, Repos, SQL Warehouses, Cluster Policies, Serverless Compute, MLflow, Lakehouse Monitoring, Auto Loader, and Delta Live Tables / Lakeflow • Expert knowledge of SQL and Python • Strong experience with BI tools such as Power BI, Tableau, or Business Objects • Strong Microsoft Azure experience, including ADLS Gen2, Microsoft Entra ID (Azure AD), RBAC, networking, and cloud security • Experience implementing Medallion Architecture, dimensional modeling, and domain-oriented data products • Deep understanding of Spark optimization techniques, including Adaptive Query Execution, partitioning, caching, Photon, Liquid Clustering, and Delta optimization • Experience implementing CI/CD pipelines, Git-based development workflows, Infrastructure as Code, and automated testing • Strong understanding of data governance, metadata management, data quality, observability, security, and compliance • Must reside and perform work within the United States • Must be willing and able to travel as needed • Must complete applicable pre-employment screening requirements • Must complete federal employment eligibility verification through Form I-9 • Preferred: experience implementing Databricks Genie, Genie Spaces, and Genie Ontologies; semantic models and AI-ready data products; enterprise BI platforms; machine learning platforms, MLOps, or Generative AI applications; dbt, Great Expectations, or similar tools; Databricks Certified Data Engineer Professional and/or Microsoft Azure certifications

🏖️ Benefits

• Medical insurance • Dental insurance • Vision insurance • Flexible spending accounts • Generous paid time off • Sponsored volunteer opportunities • 401(k) with a company match • Free access to Penn Foster Group's online programs • Remote work arrangement • Sponsored travel to team meetings, company events, conferences, and summits as applicable

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume