Senior Databricks Engineer

Job not on LinkedIn

🔥 1 hour ago

🏛️ District of Columbia – Remote

🏛️ District of Columbia – Remote

⏰ Full Time

🟠 Senior

🧑‍💻 Full-stack Engineer

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Koniag Government Services

WebsiteLinkedInAll Job Openings

1001 - 5000 employees

Founded 1975

🏛️ Government

🎖️ Defense

💼 Consulting

Government • Defense • Consulting

Koniag Government Services is an Alaska Native Corporation (ANC) that provides technical, professional, and operational expertise to the U. S. public sector. KGS supports Defense & Intelligence, Federal Civilian, and Health customers with enterprise solutions, professional services, and operations management, and emphasizes mission-focused outcomes, contracting speed (ANC direct awards), and strategic/technology partnerships. The company positions itself as a mission partner delivering people, technology, and program management to government customers.

📋 Description

• designing, developing, and deploying scalable data infrastructure while driving key data engineering initiatives • Work closely with project managers, team members, and business owners to gather requirements, communicate progress, and ensure data meets customer needs. • Should be capable of architecting and implementing scalable, highly available data pipelines using Databricks, adhering strictly to the Medallion Architecture (Bronze, Silver, Gold layers) to ensure data quality, consistency, and optimized reporting. • Establish comprehensive data quality standards and implement monitoring dashboards to enable business owners and responsible parties to track and proactively address data issues. • Write clean, efficient, and production-ready code using PySpark and SQL for complex data extraction, transformation, and loading (ETL/ELT) processes. • Design and build robust data ingestion pipelines to reliably pull from various third-party external systems using RESTful API integrations. • Implement infrastructure and pipelines as code. Utilize Databricks Asset Bundles (DABs) integrated with Azure DevOps to manage and automate deployments across Development, Staging, and Production environments. • Develop, package, and maintain custom Python libraries and modules to establish common frameworks used across multiple data projects and teams. • Apply foundational and advanced data warehousing concepts (e.g., dimensional modeling, slowly changing dimensions) to optimize data storage and retrieval for downstream analytics and business intelligence. • Monitor, troubleshoot, and optimize Databricks clusters and Spark jobs to ensure maximum efficiency and cost-effectiveness. • Securely sharing data with external BI tools (such as PowerBI or SAP BusinessObjects) and developing dashboards and capabilities natively within the Databricks environment. • Serve as a technical SME for the data engineering team, conducting code reviews, establishing best practices, and mentoring other staff members.

🎯 Requirements

• Knowledge of Object Oriented Programming (OOP) concepts and proficiency in one or more programming or scripting languages (Java, C#, Python, JavaScript, PowerShell) • Experience: 8+ years of dedicated Data Engineering experience, with at least 3+ years functioning in a senior capacity focused on the Databricks ecosystem. • Core Languages: Expert-level proficiency in PySpark and complex SQL. • Python Library Management: Proven ability to create, package, and manage custom Python libraries (e.g., building Wheel files) for scalable reuse across multiple data pipelines and projects. • API Data Ingestion: Deep understanding of HTTP REST methods (GET, POST, PUT, etc.) with hands-on experience extracting, paginating, and processing data from external API sources. • Frameworks & Methodology: Deep theoretical and practical understanding of modern Data Warehousing concepts and hands-on experience building out the Medallion Architecture. • DevOps & CI/CD: Proven experience automating Databricks deployments. You must have hands-on experience configuring and deploying Databricks Asset Bundles (DABs) using Azure DevOps (creating YAML pipelines, managing service principals, and handling multi-workspace deployments). • Cloud Platforms: Strong familiarity with the broader cloud ecosystem surrounding Databricks (e.g., Azure Data Lake Storage Gen2, Azure Key Vault). • Version Control: Proficiency with Git-based version control workflows. • Concise and comprehensive written and oral communication skills for both technically understanding and customer usage support. • Secret Security clearance

🏖️ Benefits

• health, dental and vision insurance • 401K with company matching • flexible spending accounts • paid holidays • three weeks paid time off • more