Senior Databricks Engineer

Vaga não está no LinkedIn

🕒 Ontem

🏛️ District of Columbia – Remoto

🏛️ District of Columbia – Remote

⏰ Tempo Integral

🟠 Sênior

🧑‍💻 Engenheiro Full-stack

Candidatar-se

Encontrar Vagas Remotas Similares

Configurar Alerta de Email para Novas Vagas

📊 Verifique sua pontuação de currículo para esta vaga

Melhore suas chances de conseguir uma entrevista verificando sua pontuação de currículo antes de se candidatar.

Enviar Currículo

Koniag Government Services

SiteLinkedInTodas as Vagas

1001 - 5000 funcionários

Fundada em 1975

🏛️ Governo

🎖️ Defesa

💼 Consultoria

Government • Defense • Consulting

A Koniag Government Services é uma Corporação Nativa do Alaska (ANC) que fornece expertise técnica, profissional e operacional para o setor público dos EUA. A KGS apoia os clientes de Defesa & Inteligência, Civil Federal e Saúde com soluções corporativas, serviços profissionais e gestão de operações, enfatizando resultados focados na missão, velocidade na contratação (prêmios diretos ANC) e parcerias estratégicas/tecnológicas. A empresa se posiciona como um parceiro de missão entregando pessoas, tecnologia e gestão de programas para clientes governamentais.

Descrição

• designing, developing, and deploying scalable data infrastructure while driving key data engineering initiatives • Work closely with project managers, team members, and business owners to gather requirements, communicate progress, and ensure data meets customer needs. • Should be capable of architecting and implementing scalable, highly available data pipelines using Databricks, adhering strictly to the Medallion Architecture (Bronze, Silver, Gold layers) to ensure data quality, consistency, and optimized reporting. • Establish comprehensive data quality standards and implement monitoring dashboards to enable business owners and responsible parties to track and proactively address data issues. • Write clean, efficient, and production-ready code using PySpark and SQL for complex data extraction, transformation, and loading (ETL/ELT) processes. • Design and build robust data ingestion pipelines to reliably pull from various third-party external systems using RESTful API integrations. • Implement infrastructure and pipelines as code. Utilize Databricks Asset Bundles (DABs) integrated with Azure DevOps to manage and automate deployments across Development, Staging, and Production environments. • Develop, package, and maintain custom Python libraries and modules to establish common frameworks used across multiple data projects and teams. • Apply foundational and advanced data warehousing concepts (e.g., dimensional modeling, slowly changing dimensions) to optimize data storage and retrieval for downstream analytics and business intelligence. • Monitor, troubleshoot, and optimize Databricks clusters and Spark jobs to ensure maximum efficiency and cost-effectiveness. • Securely sharing data with external BI tools (such as PowerBI or SAP BusinessObjects) and developing dashboards and capabilities natively within the Databricks environment. • Serve as a technical SME for the data engineering team, conducting code reviews, establishing best practices, and mentoring other staff members.

🎯 Requisitos

• Knowledge of Object Oriented Programming (OOP) concepts and proficiency in one or more programming or scripting languages (Java, C#, Python, JavaScript, PowerShell) • Experience: 8+ years of dedicated Data Engineering experience, with at least 3+ years functioning in a senior capacity focused on the Databricks ecosystem. • Core Languages: Expert-level proficiency in PySpark and complex SQL. • Python Library Management: Proven ability to create, package, and manage custom Python libraries (e.g., building Wheel files) for scalable reuse across multiple data pipelines and projects. • API Data Ingestion: Deep understanding of HTTP REST methods (GET, POST, PUT, etc.) with hands-on experience extracting, paginating, and processing data from external API sources. • Frameworks & Methodology: Deep theoretical and practical understanding of modern Data Warehousing concepts and hands-on experience building out the Medallion Architecture. • DevOps & CI/CD: Proven experience automating Databricks deployments. You must have hands-on experience configuring and deploying Databricks Asset Bundles (DABs) using Azure DevOps (creating YAML pipelines, managing service principals, and handling multi-workspace deployments). • Cloud Platforms: Strong familiarity with the broader cloud ecosystem surrounding Databricks (e.g., Azure Data Lake Storage Gen2, Azure Key Vault). • Version Control: Proficiency with Git-based version control workflows. • Concise and comprehensive written and oral communication skills for both technically understanding and customer usage support. • Secret Security clearance

🏖️ Benefícios

• health, dental and vision insurance • 401K with company matching • flexible spending accounts • paid holidays • three weeks paid time off • more