Lead Data Engineer

🔥 5 minutes ago

🇮🇳 India – Remote

⏰ Full Time

🟠 Senior

🚰 Data Engineer

👻 Ghost score 11%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Firmable

WebsiteLinkedInAll Job Openings

51 - 200 employees

Founded 2023

🤝 B2B

☁️ SaaS

🤖 Artificial Intelligence

B2B • SaaS • Artificial Intelligence

Firmable is an AI-native B2B sales platform that provides company and contact data, prospect list building, automated buying-signal monitoring, and CRM enrichment. It uses LLMs and agentic AI to assemble and refresh data across hundreds of sources, surface high-intent accounts (role changes, funding, search intent, technology adoption, vertical signals), and generate CRM tasks to guide timely outreach. Firmable integrates with major CRMs (HubSpot, Salesforce, Dynamics, Pipedrive), browser extensions, and other tools, and is positioned as an alternative to legacy data providers like ZoomInfo and Apollo, targeting sales leaders, account executives, SDRs, revenue operations, marketing, and recruiters. The product is offered as a SaaS with terms aimed at smaller and mid-market teams (no enterprise-only contracts or auto-renew traps) and is used by 1,300+ businesses.

📋 Description

• Architect and own the end-to-end extraction and ETL pipeline transforming unstructured web data into a B2B dataset across 13 markets • Set architectural standards for extractor patterns, proxy strategy, LLM infrastructure, agentic escalation workflows, coverage, schema, and accuracy • Design extraction, normalisation, deduplication, validation, and load processes • Own cost, performance, reliability, scheduling, incremental processing, and recovery design • Build extractor frameworks and ship difficult extractors handling anti-bot defences, JS-heavy rendering, schema drift, and low-quality structure • Build agentic extraction pipelines with rule-based triage, LLM escalation, structured-output validation, retries, and human-review queues • Establish production LLM infrastructure with versioned prompts, labelled evaluation sets, precision/recall measurement, rollback, traces, and drift detection • Develop evaluation and observability scaffolding and model-choice playbooks • Create versioned SKILL.md specifications and orchestration patterns using Airflow or equivalent • Provide technical input to data platform, product, and analytics teams • Deliver approximately 80% hands-on architecture/reference implementation and 20% cross-functional technical input

🎯 Requirements

• 6–10 years shipping production extraction, ETL, or data pipeline systems in business-critical environments • Deep web extraction at scale, including anti-bot defences, proxy architecture, JS rendering, schema drift, and recovery • Production LLMs in extraction pipelines, including structured outputs, versioned prompts, labelled eval sets, logged traces, precision/recall judges, and vendor-model drift detection • Production agent workflows and tool-calling pipelines • Experience writing SKILL.md specifications • Strong judgment regarding deterministic rules versus LLMs • Expert Python, including concurrency and scale • Advanced SQL for complex transformations and performance optimisation • Extensive Airflow or equivalent experience • Experience shipping with agentic IDEs such as Claude Code or Cursor • Architecture judgment and product mindset • Current hands-on use of AI tools, structured outputs, evals, traces, and LLM tracing • Cloud data platforms such as Snowflake or Redshift highly valued • AWS pipeline deployment experience with Lambda, S3, ECS, or Glue highly valued • Vector databases, embeddings, or retrieval patterns highly valued • Eval frameworks such as Braintrust, Promptfoo, or Inspect highly valued • Data-quality frameworks with automated testing and anomaly detection highly valued • B2B data experience highly valued • Data privacy and compliance knowledge, including GDPR and CCPA, highly valued • Startup or scaleup experience highly valued

🏖️ Benefits

• No fixed hours • Full autonomy on architecture choices