Data Engineering Manager

Job not on LinkedIn

🔥 1 minute ago

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Penn Foster

WebsiteLinkedInAll Job Openings

201 - 500 employees

📚 Education

👥 B2C

🤝 B2B

Education • B2C • B2B

Penn Foster is an accredited, career-focused online education provider offering self-paced certificate, diploma, associate, bachelor's degree and high school programs. Its catalog covers vocational and professional areas including healthcare, veterinary technology, skilled trades (HVACR, electrical, plumbing), automotive repair, business, IT, criminal justice, design, hospitality, and more. Penn Foster serves individual learners (over 133,000 active students) with flexible, 24/7 online coursework and also provides employer-facing training solutions. The institution operates multiple schools/divisions and maintains accreditations such as DEAC and program-specific accreditations (for example, an AVMA-CVTEA-accredited veterinary technician program).

📋 Description

• Provide technical leadership for the Data Engineering team through architecture guidance, code reviews, mentoring, and engineering best practices • Mentor and develop junior and mid-level Data Engineers • Establish engineering standards for software development, testing, CI/CD, documentation, observability, and operational excellence • Lead technical design discussions, evaluate architectural tradeoffs, and drive adoption of modern engineering practices • Serve as the technical lead for Penn Foster Group's Databricks Lakehouse platform • Design, build, and support scalable enterprise data pipelines using SQL, Python, Apache Spark, and Databricks • Define Databricks development best practices across Workflows, Repos, Jobs, notebooks, Python libraries, Git integration, cluster policies, SQL Warehouses, and deployment automation • Design and optimize Delta Lake architectures using Medallion patterns, Delta optimization, Liquid Clustering, and Photon • Build reusable ingestion frameworks for batch, streaming, CDC, and API-based integrations • Optimize Spark workloads for performance, scalability, reliability, and cloud cost efficiency • Develop trusted semantic data products, Genie Spaces, and Genie Ontologies for Databricks Genie • Establish best practices for semantic modeling, governed metrics, business metadata, and AI-ready datasets • Evaluate and implement emerging Databricks AI capabilities • Collaborate with Data Science and MLOps teams on machine learning, generative AI, and advanced analytics initiatives • Design scalable Lakehouse architectures supporting analytics, reporting, AI, and operational data products • Implement enterprise governance using Unity Catalog, including data lineage, fine-grained security, metadata management, and access controls • Champion automated testing, monitoring, observability, data quality, and production reliability • Serve as the technical escalation point for complex production issues and lead root cause analysis • Drive platform modernization initiatives while balancing delivery, scalability, maintainability, and operational excellence • Collaborate with Business Intelligence, Product, Platform Engineering, Security, and MLOps teams • Translate business requirements into technical solutions for enterprise reporting, analytics, AI, and strategic decision-making • Contribute to technical roadmaps, platform strategy, and the evolution of Data & Analytics capabilities

🎯 Requirements

• Bachelor's degree in Computer Science, Engineering, Information Systems, or a related field, or equivalent practical experience • 8+ years of experience in Data Engineering, Analytics Engineering, or Data Platform Engineering • 3+ years of experience leading technical initiatives and mentoring engineering teams • Demonstrated success developing engineers while driving engineering excellence and technical standards • Strong communication and collaboration skills with technical and business stakeholders • Expert knowledge of the Databricks Lakehouse Platform, including Apache Spark/PySpark, Delta Lake, Unity Catalog, Workflows & Jobs, Repos, SQL Warehouses, Cluster Policies, Serverless Compute, MLflow, Lakehouse Monitoring, Auto Loader, and Delta Live Tables/Lakeflow • Expert knowledge of SQL and Python • Strong experience with BI tools such as Power BI, Tableau, or Business Objects • Strong Microsoft Azure experience, including ADLS Gen2, Microsoft Entra ID/Azure AD, RBAC, networking, and cloud security • Experience implementing Medallion Architecture, dimensional modeling, and domain-oriented data products • Deep understanding of Spark optimization techniques, including Adaptive Query Execution, partitioning, caching, Photon, Liquid Clustering, and Delta optimization • Experience implementing CI/CD pipelines, Git-based development workflows, Infrastructure as Code, and automated testing • Strong understanding of data governance, metadata management, data quality, observability, security, and compliance • Candidates must complete a role-specific assessment as the first step in the hiring process • Successful completion of applicable pre-employment screening requirements • Completion of federal employment eligibility verification through Form I-9 • Preferred: experience with Databricks Genie, Genie Spaces, and Genie Ontologies • Preferred: experience designing semantic models and AI-ready data products • Preferred: experience supporting enterprise BI platforms • Preferred: experience with machine learning platforms, MLOps, or Generative AI applications • Preferred: experience with dbt, Great Expectations, or similar tools • Preferred: Databricks Certified Data Engineer Professional and/or Microsoft Azure certifications

🏖️ Benefits

• Medical insurance • Dental insurance • Vision insurance • Flexible spending • Generous paid time off • Sponsored volunteer opportunities • 401K with a company match • Free access to all of Penn Foster Group's online programs • Remote work arrangement • On-camera work environment with remote collaboration