Senior Databricks Engineer

Emploi pas sur LinkedIn

🕒 il y a 10 jours

🏛️ District of Columbia – Distant

🏛️ District of Columbia – Remote

⏰ Temps Plein

🟠 Senior

👷🏻‍♀️ Ingénieur

Postuler Maintenant

Trouver des Emplois à Distance Similaires

Configurer une alerte email pour les nouveaux emplois

📊 Vérifiez votre score de CV pour ce poste

Améliorez vos chances d'obtenir un entretien en vérifiant votre score de CV avant de postuler.

Télécharger le CV

Koniag Government Services

Site WebLinkedInTous les Emplois

1001 - 5000 employés

Fondée en 1975

🏛️ Gouvernement

🎖️ Défense

💼 Conseil

Government • Defense • Consulting

Koniag Government Services est une société indigène d'Alaska (ANC) qui fournit une expertise technique, professionnelle et opérationnelle au secteur public américain. KGS soutient les clients de la Défense et du Renseignement, les civils fédéraux, et ceux de la Santé avec des solutions d'entreprise, des services professionnels et une gestion des opérations. Elle met l'accent sur des résultats axés sur la mission, une rapidité de contractualisation (récompenses directes ANC) et des partenariats stratégiques/technologiques. L'entreprise se positionne comme un partenaire de mission fournissant du personnel, de la technologie et de la gestion de programmes à ses clients gouvernementaux.

Description

• designing, developing, and deploying scalable data infrastructure while driving key data engineering initiatives • Work closely with project managers, team members, and business owners to gather requirements, communicate progress, and ensure data meets customer needs. • Should be capable of architecting and implementing scalable, highly available data pipelines using Databricks, adhering strictly to the Medallion Architecture (Bronze, Silver, Gold layers) to ensure data quality, consistency, and optimized reporting. • Establish comprehensive data quality standards and implement monitoring dashboards to enable business owners and responsible parties to track and proactively address data issues. • Write clean, efficient, and production-ready code using PySpark and SQL for complex data extraction, transformation, and loading (ETL/ELT) processes. • Design and build robust data ingestion pipelines to reliably pull from various third-party external systems using RESTful API integrations. • Implement infrastructure and pipelines as code. Utilize Databricks Asset Bundles (DABs) integrated with Azure DevOps to manage and automate deployments across Development, Staging, and Production environments. • Develop, package, and maintain custom Python libraries and modules to establish common frameworks used across multiple data projects and teams. • Apply foundational and advanced data warehousing concepts (e.g., dimensional modeling, slowly changing dimensions) to optimize data storage and retrieval for downstream analytics and business intelligence. • Monitor, troubleshoot, and optimize Databricks clusters and Spark jobs to ensure maximum efficiency and cost-effectiveness. • Securely sharing data with external BI tools (such as PowerBI or SAP BusinessObjects) and developing dashboards and capabilities natively within the Databricks environment. • Serve as a technical SME for the data engineering team, conducting code reviews, establishing best practices, and mentoring other staff members.

🎯 Exigences

• Knowledge of Object Oriented Programming (OOP) concepts and proficiency in one or more programming or scripting languages (Java, C#, Python, JavaScript, PowerShell) • Experience: 8+ years of dedicated Data Engineering experience, with at least 3+ years functioning in a senior capacity focused on the Databricks ecosystem. • Core Languages: Expert-level proficiency in PySpark and complex SQL. • Python Library Management: Proven ability to create, package, and manage custom Python libraries (e.g., building Wheel files) for scalable reuse across multiple data pipelines and projects. • API Data Ingestion: Deep understanding of HTTP REST methods (GET, POST, PUT, etc.) with hands-on experience extracting, paginating, and processing data from external API sources. • Frameworks & Methodology: Deep theoretical and practical understanding of modern Data Warehousing concepts and hands-on experience building out the Medallion Architecture. • DevOps & CI/CD: Proven experience automating Databricks deployments. You must have hands-on experience configuring and deploying Databricks Asset Bundles (DABs) using Azure DevOps (creating YAML pipelines, managing service principals, and handling multi-workspace deployments). • Cloud Platforms: Strong familiarity with the broader cloud ecosystem surrounding Databricks (e.g., Azure Data Lake Storage Gen2, Azure Key Vault). • Version Control: Proficiency with Git-based version control workflows. • Concise and comprehensive written and oral communication skills for both technically understanding and customer usage support. • Secret Security clearance

🏖️ Avantages

• health, dental and vision insurance • 401K with company matching • flexible spending accounts • paid holidays • three weeks paid time off • more