Databricks Practice Lead – Engineering Manager

Job not on LinkedIn

🕒 2 days ago

🇺🇸 United States – Remote

⏳ Contract/Temporary

🟠 Senior

👮‍♀️ Software Engineering Manager

🦅 H1B Visa Sponsor

This company has sponsored H1B visas in the past.

👻 Ghost score 10%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Scicom Infrastructure Services

WebsiteLinkedInAll Job Openings

11 - 50 employees

🏢 Enterprise

☁️ SaaS

🤝 B2B

Enterprise • SaaS • B2B

Scicom Infrastructure Services is an industry-leading technology consulting and services organization. The company is focused on delivering high-quality, reliable, and cost-effective technology solutions to support the business objectives of leading private and public sector organizations. Scicom provides a range of services including enterprise performance management, managed services, enterprise architecture, project management, infrastructure management, and application management. They collaborate with companies like AppDynamics, BigPanda, DataDog, and AWS for business and application performance, event correlation and automation, enterprise monitoring, cloud monitoring, and ITSM solutions. Scicom offers global, 24/7 operational support and is recognized for its ability to support global infrastructure effectively. The company has been named one of the fastest-growing private companies in America by Inc 5000 in multiple years.

📋 Description

• Serve as the organization’s subject-matter expert for the Databricks Lakehouse Platform • Design scalable, secure, and highly available data architectures using Databricks, Apache Spark, Delta Lake, and cloud-native technologies • Lead batch, streaming, ETL, ELT, analytics, machine-learning, and AI-enabled data solutions • Define architectural standards for medallion architectures, data modeling, ingestion, transformation, orchestration, and data consumption • Establish Unity Catalog governance frameworks covering lineage, access controls, auditing, metadata management, and secure data sharing • Guide workspace design, cluster configuration, serverless computing, workload isolation, performance tuning, and cost optimization • Oversee Databricks integration with Microsoft Azure, AWS, or Google Cloud • Develop or review solutions involving PySpark, Spark SQL, Python, Delta Live Tables, Structured Streaming, Auto Loader, MLflow, and Databricks Workflows • Lead platform migrations and modernization from legacy databases, data warehouses, Hadoop, and traditional ETL platforms • Establish standards for source control, automated testing, CI/CD, infrastructure as code, monitoring, and production support • Conduct architecture reviews, code reviews, technical assessments, and root-cause analyses • Manage, mentor, and develop Databricks engineers, data engineers, architects, and technical consultants • Assign resources, establish goals and performance expectations, conduct reviews and coaching, and support recruiting and workforce planning • Develop reusable accelerators, reference architectures, templates, and delivery playbooks • Build and maintain a Databricks practice supporting multiple concurrent client engagements • Provide delivery oversight from project planning through implementation and operational support • Translate business, functional, security, and contractual requirements into technical plans and deliverables • Develop estimates, staffing plans, schedules, milestones, and risk-mitigation strategies • Monitor scope, schedule, quality, budget, resource utilization, dependencies, and technical risks • Coordinate engineering, cloud, cybersecurity, governance, analytics, project-management, and client teams • Track delivery metrics and provide status reports to leadership, clients, and stakeholders • Lead technical escalations and support statements of work, proposals, estimates, presentations, demonstrations, and client meetings

🎯 Requirements

• Bachelor’s degree in computer science, information technology, data engineering, engineering, or a related discipline • At least 10 years of experience in data engineering, data architecture, analytics engineering, or related technology roles • At least 5 years of hands-on experience designing and implementing solutions using Databricks • At least 3 years of experience managing or formally leading technical engineering teams • Advanced experience with Databricks Lakehouse Platform, Apache Spark, PySpark, Spark SQL, advanced SQL development, Delta Lake, medallion architecture, Unity Catalog, enterprise data governance, ETL and ELT pipeline architecture, batch and real-time data processing, data modeling, data warehousing, Python-based data engineering, Databricks Workflows, Jobs, and cluster management • Experience deploying Databricks solutions in Azure, AWS, or Google Cloud • Experience with CI/CD, Git-based development, automated testing, and infrastructure as code • Demonstrated ability to optimize Spark workloads, cluster configurations, query performance, reliability, and cloud costs • Experience managing technical delivery, resource assignments, risks, schedules, and client expectations • Strong written, verbal, presentation, documentation, and stakeholder-management skills • Ability to explain complex technical concepts to executives, business stakeholders, and nontechnical audiences • Preferred: Databricks or cloud professional certifications; consulting, professional-services, systems-integration, or managed-services experience; government-client experience; federal security, privacy, governance, and compliance knowledge; experience with Azure Data Factory, Azure Data Lake Storage, Azure Synapse Analytics, AWS Glue, Amazon S3, Snowflake, dbt, Kafka, Airflow, Terraform, MLflow, MLOps, generative AI, Databricks Mosaic AI, vector search, or machine-learning deployment; familiarity with data standards and metadata frameworks; experience managing geographically distributed or remote teams; proposal and statement-of-work experience

🏖️ Benefits

• No benefits, perks, or compensation extras are specified in the posting