Senior Data Engineer

🔥 13 hours ago

⚔️ Virginia – Remote

⚔️ Virginia – Remote

💵 $156.4k - $179.2k / year

⏰ Full Time

🟠 Senior

🚰 Data Engineer

👻 Ghost score 0%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Systems Planning & Analysis

WebsiteLinkedInAll Job Openings

1001 - 5000 employees

📦 Logistics

🏥 Healthcare

💼 Consulting

Logistics • Healthcare • Consulting

Systems Planning & Analysis is dedicated to enhancing national security through innovative data-driven analysis and technical engineering services. With a rich history of over 50 years, SPA addresses complex challenges faced by government clients, particularly in defense intelligence and homeland security. The company excels in areas such as AI/ML and decision analytics, systems engineering, and modeling and simulation, providing vital support for national security programs across multiple domains.

📋 Description

• Design, develop, and maintain scalable, production-ready data pipelines and data products using Spark (Python/SQL) in a Databricks environment • Lead integration and transformation of complex data from diverse DoW and federal health systems and sources • Design scalable data ingestion, integration, and exchange approaches, including API-based integrations and services • Establish and promote reusable data engineering patterns, standards, and best practices • Provide technical guidance on data architecture, pipeline design, data modeling, integration approaches, and engineering practices • Monitor, troubleshoot, and optimize production workflows and data pipelines • Identify performance, reliability, and scalability improvements • Define and implement data validation, quality, and governance practices • Troubleshoot complex technical and data integration challenges, identify root causes, and drive sustainable solutions • Collaborate with engineers, architects, analysts, and customer stakeholders to translate data needs into scalable technical solutions • Mentor other engineers and help teams navigate complex technical challenges • Improve engineering tools, processes, and patterns and drive their adoption • Own complex technical areas and maintain engineering quality, consistency, and cohesion as the platform and data product portfolio grows

🎯 Requirements

• Active Secret clearance / active DoW Secret clearance • Master’s degree in Computer Science, Information Systems, Software Engineering, or a related field • 10+ years of data engineering, software engineering, or related technical experience • Extensive hands-on experience designing, building, and operating production data pipelines and data products • Advanced proficiency with Python and SQL • Strong experience with Apache Spark and distributed data processing • Experience with Databricks or similar modern data platforms • Experience designing and maintaining ETL/ELT processes for complex, large-scale datasets • Experience integrating data across disparate systems and consuming or developing API-based data integrations • Strong understanding of data modeling, data architecture, data quality, and data governance principles • Experience troubleshooting and optimizing complex production data pipelines for performance, reliability, and scalability • Experience with Git-based development workflows and modern software engineering practices • Experience working in terminal / command-line environments • Demonstrated experience providing technical guidance, mentoring engineers, and influencing engineering practices • Strong client and stakeholder communication skills • Ability to translate technical concepts for technical and non-technical audiences • Ability to independently navigate ambiguity, identify technical risks, and drive complex engineering challenges toward resolution • Experience spotting security, privacy, and compliance issues and working with security/compliance/legal stakeholders • AWS cloud experience • Experience with very large datasets, including datasets with billions of records • Experience with Palantir Foundry • GitLab experience • Experience with Advana or similar DoW data environments • Experience with federal health, financial, or other regulated and sensitive data • Experience using AI/ML to accelerate work, including automating routine tasks, accelerating development, and debugging

🏖️ Benefits

• Competitive base pay • Variable compensation opportunities • Health insurance • Flexible spending accounts • Health savings accounts • Retirement savings plans • Life and disability insurance programs • Paid and unpaid time away from work • Industry-leading 401k contribution • Collaborative, dynamic work environment