Open Source Engineer

October 25

Apply Now

Receive Emails with Similar Jobs

LanceDB

WebsiteLinkedInAll Job Openings

Artificial Intelligence • Enterprise • SaaS

LanceDB is the first open-source AI-Native Multimodal Lakehouse designed to handle complex AI data needs on a large scale. It offers a unified solution for multimodal data, integrating storage, search, feature engineering, and analytics into a single platform. This enables users to efficiently manage diverse data types and workflows, facilitating rapid iteration and deployment of AI models. With advanced capabilities such as hybrid search and optimized training pipelines, LanceDB is built for enterprises seeking to leverage their data effectively in AI applications.

11 - 50 employees

Founded 2022

🤖 Artificial Intelligence

🏢 Enterprise

☁️ SaaS

📋 Description

• Driving open-source community efforts to integrate the Lance format with Spark, Hive Metastore, Presto, Trino, Ray, and other data infrastructure systems • Designing and maintaining efficient distributed Lance dataset operations • Building efficient indices to enable predicate pushdown and accelerate queries in Spark, Ray, or Trino • Working on table formats, data encodings, and various aspects of the Lance format in Rust • Operating and improving internal data processing infrastructure • Promoting the Lance format in open-source communities and at Big Data conferences

🎯 Requirements

• 5+ years of experience building high-performance databases, big data systems, or large-scale data services • Deep understanding of internals of open-source Big Data or AI training systems (e.g., Hadoop, Spark, Flink, Ray, Iceberg, Delta Lake, Hudi, ClickHouse, Trino, Presto, PyTorch, or JAX) • Strong experience with high-performance computing in Java or Scala • Experience with Rust (or willingness to learn it) • Proven ability to move fast, work independently, and collaborate with a high-caliber team

🏖️ Benefits

• Medical, dental, vision, and life insurance • 401(k) retirement plan • Flexible Spending Accounts (FSA) and Health Savings Accounts (HSA) • Commuter benefits • Generous paid time off