Senior Staff Engineer – Data Lakehouse Platform

October 7

🗽 New York – Remote

Although this job is listed in New York, this company may be open to hiring remote in other states.

🦀 Maryland – Remote

Although this job is listed in Maryland, this company may be open to hiring remote in other states.

+1 more states

🗽 New York – Remote 🦀 Maryland – Remote 🤠 Texas – Remote

💵 $110k - $260k / year

⏰ Full Time

🟠 Senior

🧑‍💻 Full-stack Engineer

🦅 H1B Visa Sponsor

Apply Now

Receive Emails with Similar Jobs

GEICO

WebsiteLinkedInAll Job Openings

Finance • Insurance

GEICO is a well-known American auto insurance company, providing coverage to a wide range of customers. As a leading provider of car insurance, GEICO is known for its innovative marketing and customer-centric services, delivering affordable insurance options. The company utilizes a user-friendly website for managing policies and claims, applying necessary online privacy measures to protect personal data and improve user experience. While primarily focused on auto insurance, GEICO also offers other types of insurance, including homeowner and renter policies.

10,000+ employees

Founded 1936

💸 Finance

📋 Description

• Scope, design, and build scalable, resilient Data Lakehouse components. • Lead architecture sessions and reviews with peers and leadership. • Spearhead new software evaluations and innovate with new tooling. • Design and lead the development & implementation of Compute Efficiency projects like Smart Spark Auto-Tuning Feature. • Drive performance regression testing, benchmarking, and continuous performance profiling. • Accountable for the quality, usability, and performance of the solutions. • Determine and support resource requirements, evaluate operational processes, measure outcomes to ensure desired results, and demonstrate adaptability and sponsoring continuous learning. • Collaborate with customers, team members, and other engineering teams to solve our toughest problems. • Be a role model and mentor, helping to coach and strengthen the technical expertise and know-how of our engineering community. • Consistently share best practices and improve processes within and across teams. • Share your passion for staying on top of the latest open-source projects, experimenting with, and learning recent technologies, participating in internal and external OSS technology communities, and mentoring other members of the engineering community.

🎯 Requirements

• Deep knowledge of Spark internals, including Catalyst, Tungsten, AQE, CBO, scheduling, shuffle management, and memory tuning. • Proven experience in tuning and optimizing Spark jobs on Hyper-Scale Spark Compute Platforms. • Mastery of Spark configuration parameters, resource tuning, partitioning strategies, and job execution behaviors. • Experience building automated optimization systems – from config auto-tuners to feedback loops and adaptive pipelines. • Strong software engineering skills in Scala, Java, and python are required. • Ability to build tooling to surface meaningful performance insights at scale. • Deep understanding of auto-scaling and cost-efficiency strategies in cloud-based Spark environments. • Exemplary ability to design and develop, perform experiments, and influence engineering direction and product roadmap. • Advanced experience developing new and enhancing existing open-source based Data Lakehouse platform components. • Experience cultivating relationships with and contributing to open-source software projects. • Experience with open-source table formats (Apache Iceberg, Delta, Hudi or equivalent). • Advanced experience with open-source compute engines (Apache Spark, Apache Flink, Trino/Presto, or equivalent). • Experience with cloud computing (AWS, Microsoft Azure, Google Cloud, Hybrid Cloud, or equivalent). • Expertise in developing distributed systems that are scalable, resilient, and highly available. • Experience in container technology like Docker and Kubernetes platform development. • Experience with continuous delivery and infrastructure as code. • In-depth knowledge of DevOps concepts and cloud architecture. • Experience in Azure Network (Subscription, Security zoning, etc.) or equivalent.

🏖️ Benefits

• Comprehensive Total Rewards program that offers personalized coverage tailor-made for you and your family’s overall well-being. • Financial benefits including market-competitive compensation; a 401K savings plan vested from day one that offers a 6% match; performance and recognition-based incentives; and tuition assistance. • Access to additional benefits like mental healthcare as well as fertility and adoption assistance. • Supports flexibility- We provide workplace flexibility as well as our GEICO Flex program, which offers the ability to work from anywhere in the US for up to four weeks per year.