Senior Lead Data Engineer

🔥 1 minute ago

🇺🇸 United States – Remote

💵 $50 - $75 / hour

⏱ Part Time

🟠 Senior

🚰 Data Engineer

🦅 H1B Visa Sponsor

This company has sponsored H1B visas in the past.

👻 Ghost score 1%

What it means: the chance that no one is really hiring for this job.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.Why this score:

• Just posted.
• Jobs at this company usually fill in 29 days.

Apply Now

Find Similar Remote Jobs

Lighthouse

WebsiteLinkedInAll Job Openings

501 - 1000 employees

Founded 1995

💼 Consulting

🏥 Healthcare

📦 Logistics

💰 $23M Debt Financing on 2016-08

Consulting • Healthcare • Logistics

Lighthouse is a company that specializes in eDiscovery and information governance, employing advanced AI technology to streamline legal processes and improve outcomes for legal teams. They offer a range of services including document review, forensic data collection, and strategic consulting, with a strong focus on reducing costs and optimizing data management strategies. Lighthouse partners with various industries, including financial services and healthcare, to help manage complex data environments effectively while ensuring compliance and minimizing risk.

📋 Description

• Set the technical direction and own end-to-end data engineering architecture for Prism eDiscovery AI products • Lead the design, build, and management of preprocessing, validation, and transformation pipelines for heterogeneous ESI corpora at multi-million document scale • Lead Data Orchestration platforms coordinating training data preparation, model scoring, embedding generation, and result delivery • Architect and implement pipelines for BERT fine-tuning and LLM context preparation • Direct infrastructure for LLM and Generative AI applications, including document chunking, embedding generation, vector database population, and RAG retrieval optimization • Define and drive data engineering standards, best practices, design patterns, code quality, testing, documentation, and design reviews • Establish tooling to monitor, debug, and analyze pipeline health, data quality, and model-serving data flows in production • Lead resolution of complex production issues • Design and govern scalable schemas for document metadata, model inputs and outputs, scoring results, and embeddings • Lead experiments evaluating efficiency, throughput, and cost of data engineering stacks for legal AI workloads • Make buy-versus-build recommendations for data infrastructure and present them to engineering and product leadership • Partner with data science, product, and engineering stakeholders to shape requirements, sequence technical work, and align priorities with business goals • Mentor and provide technical guidance to data engineers • Perform other related duties as assigned

🎯 Requirements

• Degree in computer science or computer engineering with 8+ years of experience in a data engineering related field, including experience leading the technical design of large-scale data platforms (required) • Demonstrated technical leadership: setting architecture direction, leading design reviews, and guiding engineers on complex, cross-team initiatives • Expertise building data pipelines for unstructured document data, including text extraction, OCR processing, metadata extraction, and legal document format handling • Experience with big data workflows in cloud-native technology stacks, preferably Azure • Experience with Spark, Databricks, Airflow, Azure Data Factory, Kafka, and Kubernetes • Deep knowledge of data science and ML pipeline patterns, including training data preparation, feature engineering, embedding generation, and MLOps operationalization • Experience with vector databases and semantic search infrastructure such as Azure AI Search or Pinecone for RAG pipeline support • Experience with relational and NoSQL databases, cloud-native storage technologies, and data serialization formats including Parquet, Delta Lake, and JSON • Experience working with data scientists to operationalize encoder-based classification models and LLM applications in production • Strong communication skills to explain complex technical trade-offs to technical and non-technical stakeholders • Proficiency with agile development methodologies • Ability to work in a typical office environment at a desk or computer table • Ability to use a computer, communicate over the telephone, and read printed material • May be required to be on call periodically and work outside normal working hours, including evenings and weekends

🏖️ Benefits

• Part-time schedule of 24 hours per week • Eligibility to participate in an annual bonus or incentive program • Equal Opportunity Employer • Commitment to diversity, inclusion, respect, and a safe workplace

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume