Senior Data Engineer

🔥 13 hours ago

⚔️ Virginia – Remote

⚔️ Virginia – Remote

💵 $156.4k - $179.2k / year

⏰ Full Time

🟠 Senior

🚰 Data Engineer

👻 Ghost score 0%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Systems Planning & Analysis

WebsiteLinkedInAll Job Openings

1001 - 5000 employees

📦 Logistics

🏥 Healthcare

💼 Consulting

Logistics • Healthcare • Consulting

Systems Planning & Analysis is dedicated to enhancing national security through innovative data-driven analysis and technical engineering services. With a rich history of over 50 years, SPA addresses complex challenges faced by government clients, particularly in defense intelligence and homeland security. The company excels in areas such as AI/ML and decision analytics, systems engineering, and modeling and simulation, providing vital support for national security programs across multiple domains.

📋 Description

• Design, develop, and maintain scalable, production-ready data pipelines and data products using Spark (Python/SQL) in a Databricks environment • Lead the integration and transformation of complex data from diverse DoW and federal health systems and sources into reliable, reusable data products • Design scalable approaches for data ingestion, integration, and exchange, including API-based integrations and services • Establish and promote reusable data engineering patterns, standards, and best practices • Provide technical guidance on data architecture, pipeline design, data modeling, integration approaches, and engineering practices • Monitor, troubleshoot, and optimize production workflows and data pipelines • Define and implement data validation, quality, and governance practices • Troubleshoot complex technical and data integration challenges, identify root causes, and drive sustainable solutions • Collaborate with engineers, architects, analysts, and customer stakeholders to translate complex data needs into scalable technical solutions • Provide technical guidance and mentorship to other engineers • Identify opportunities to improve engineering tools, processes, and patterns and help drive adoption • Take ownership of complex technical areas and maintain engineering quality, consistency, and cohesion as the platform and portfolio of data products grow

🎯 Requirements

• Master’s degree in Computer Science, Information Systems, Software Engineering, or a related field • 10+ years of data engineering, software engineering, or related technical experience • Extensive hands-on experience designing, building, and operating production data pipelines and data products • Advanced proficiency with Python and SQL • Strong experience with Apache Spark and distributed data processing • Experience working with Databricks or similar modern data platforms • Experience designing and maintaining ETL/ELT processes for complex, large-scale datasets • Experience troubleshooting and optimizing complex production data pipelines for performance, reliability, and scalability • Experience working with Git-based development workflows and modern software engineering practices • Demonstrated experience providing technical guidance, mentoring engineers, and influencing engineering practices • Strong client and stakeholder communication skills, with the ability to translate technical concepts and recommendations for both technical and non-technical audiences • Ability to independently navigate ambiguity, identify technical risks, and drive complex engineering challenges toward resolution • Experience spotting security, privacy and compliance issues and working with security/compliance/legal stakeholders • Possess and maintain an active Secret clearance • AWS cloud experience • Experience working with very large datasets, including datasets with billions of records • Experience with Palantir Foundry • GitLab experience • Experience working with Advana or similar DoW data environments • Experience working with federal health, financial, or other regulated and sensitive data • Experience using AI/ML to accelerate work, including automating routine tasks, accelerating development and debugging

🏖️ Benefits

• Competitive base pay • Variable compensation opportunities • Health insurance • Flexible spending accounts • Health savings accounts • Retirement savings plans • Life and disability insurance programs • Paid and unpaid time away from work