Senior Data Engineer – Design, Architecture

🕒 Junho 2

Candidatar-se

Encontrar Vagas Remotas Similares

Receber Emails de Vagas Remotas

📊 Verifique sua pontuação de currículo para esta vaga

Melhore suas chances de conseguir uma entrevista verificando sua pontuação de currículo antes de se candidatar.

Enviar Currículo

K2United

SiteLinkedInTodas as Vagas

51 - 200 funcionários

Quando a Knowledge2Share foi criada há 22 anos, a startup foi impulsionada por indivíduos apaixonados que valorizavam o impulso interno para inovar. Desde então, a empresa evoluiu para duas empresas: K2Share, uma consultoria de cibersegurança, e CareerSafe, líder em treinamentos online OSHA e em educação voltada para a carreira. Enquanto nossas marcas individuais, K2Share e CareerSafe, representam o que fazemos (e fazemos muito bem, diga-se de passagem!), nenhuma delas representa completamente quem somos. No nosso cerne, somos exatamente o que nosso fundador, Dr. Larry Teverbaugh, pretendia criar: um ótimo lugar para trabalhar, onde os funcionários podem se divertir enquanto causam impacto. Sob nossa nova organização, K2United, nossas duas empresas podem continuar a crescer e evoluir, unidas pelo que nos une: nosso propósito. Juntos, criamos soluções para que aqueles que servimos prosperem.

Descrição

• The Senior Data Engineer will own the data engineering function on K2Share's Federal Team, partnering with technical and product leadership to deliver data products that support mission-critical decision-making for federal agency clients. • Design and build relational data layers that handle OSCAL and other structured compliance data - including ingestion, validation, transformation, and export workflows that preserve fidelity to source schemas across the full data lifecycle • Design and maintain data models that support governance, risk, compliance, scoring, and reporting workflows for federal cybersecurity programs, with OSCAL as the connective layer across them - including long-term retention and archival policies that align with federal recordkeeping and audit requirements • Design and build big-data processing pipelines on Databricks (PySpark, Delta Lake, Unity Catalog) that normalize cybersecurity data from across federal agency environments and produce analytical layers for trend analysis, executive reporting, and cross-program insights • Optimize data systems for performance and cost - identifying I/O and compute bottlenecks, scaling compute responsibly, and balancing throughput against the cost discipline federal engagements require • Architect, build and maintain AWS data infrastructure that meets federal security and operational requirements - working across services such as S3, Bedrock, Lambda, Fargate, and EC2 in support of compliance and analytical workloads • Design and implement audit-ready data primitives - change capture, access controls, validation, and lineage — that support agency reporting and continuous monitoring needs • Lead AI-first development and responsible AI deployment on the data team - using AI development tools as a standard part of the engineering loop, prototyping AI-assisted compliance workflows, and designing the production AI systems behind them (RAG architectures, vector store management, conversational agents, prompt and output guardrails, and evaluation pipelines), in alignment with federal AI governance guidance (OMB, NIST AI RMF) • Engage with federal agency stakeholders and internal teams during requirement discovery, delivery, and ongoing support — translating compliance needs into data products and customer feedback into improvements

🎯 Requisitos

• 5+ years of production data engineering experience, with a track record of designing and owning data systems end-to-end • Strong relational database expertise with PostgreSQL or equivalent — schema design at scale, indexing and partitioning strategy, access control, and patterns for handling semi-structured data including JSON. • Strong system design and architecture instincts - able to translate business and compliance requirements into data system designs, document tradeoffs, and lead design reviews with technical and non-technical stakeholders. • 3+ years of big-data processing on Databricks, Spark, or equivalent — PySpark, Delta Lake, Unity Catalog, and medallion (bronze/silver/gold) architecture patterns • Strong AWS experience including S3, Bedrock, Lambda, Fargate, EC2, relational database services, change-data-capture services, serverless compute, IAM, and KMS — ideally in GovCloud or other regulated-cloud environments • Strong Python development skills, including data manipulation libraries such as pandas, for ETL, transformation, and analytical workflows • Proficient with Git and modern version-control practices — branching strategies, code review discipline, and collaborative workflows in a team setting • Experience working with structured external schemas - OSCAL or similar standards-based data — including the discipline of preserving fidelity through transformation • Demonstrated focus on data system optimization - identifying I/O bottlenecks, remediating performance issues, balancing cost and performance, and scaling compute responsibly • Schema evolution discipline - migration strategy, backward compatibility, change-data-capture-friendly design, and the operational rigor of running production schemas under change control • Experience with async data processing patterns — task queues, message-based pipelines, idempotent task design • Active use of AI development tools as a routine part of the engineering workflow, with informed views on where they accelerate and where they need supervision • Familiarity with responsible AI deployment patterns — RAG architectures, vector databases, embedding management, prompt and output guardrails, and evaluation methods • Working knowledge of federal cybersecurity frameworks: FISMA, NIST RMF, NIST SP 800-53, NIST CSF • Demonstrated ability to interpret regulatory and policy guidance and translate it into product or data-product requirements • Comfort working across ambiguous, fast-moving federal programs with minimal supervision and strong collaborative instincts

🏖️ Benefícios

• 401(k) plan with employer matching contributions • Low-cost, comprehensive medical benefits for employees and their families • Flexibility for those needing time off for jury duty, voting, military leave, etc. • Paid time off • Wellness stipend program (includes fitness reimbursement program) • Tuition stipend • Casual dress work environment • Technical training and certifications as required • Any of our CareerSafe Online training courses for free to employees and their immediate family