Data Engineer, AI – Distributed Systems

🔥 9 minutes ago

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Zignal Labs

WebsiteLinkedInAll Job Openings

51 - 200 employees

Founded 2011

🎖️ Defense

💼 Consulting

📦 Logistics

💰 $15M Debt Financing - Zignal Labs on 2019-01

Defense • Consulting • Logistics

Zignal Labs is a technology company that provides an agentic intelligence platform for national security and defense users. Its platform delivers persistent, analyst-directed workflows that collect and enrich multimodal signals at scale, produce structured intelligence outputs and alerts via APIs, and enable earlier warning and sustained decision advantage for protecting people, facilities, and missions across political, military, economic, social, information, infrastructure and physical environments.

📋 Description

• Develop and operate batch and streaming pipelines that ingest and enrich high-volume unstructured data • Own pipeline components end to end, from implementation through production monitoring • Build and extend data pathways feeding NLP, LLM, and retrieval services • Perform data preparation, embedding generation, and indexing workflows • Integrate with and tune search and vector stores for semantic search, clustering, and real-time retrieval • Implement and improve microservices and APIs delivering analytics to enterprise customers • Write clean, tested, maintainable code • Participate in code review, CI/CD, and infrastructure-as-code practices • Debug production issues and improve reliability • Collaborate with Data Science, ML, Product, and Security teams to move ideas from prototype into production

🎯 Requirements

• 3+ years building and operating data pipelines in production • Strong programming skills in a JVM language: Scala, Java, or Kotlin • Working proficiency in Python for data and scripting work • Hands-on experience with a distributed processing framework, most likely Apache Spark • Hands-on experience with a streaming platform, most likely Kafka, including consumer groups, offsets, partitioning, and lag handling • Practical AWS experience and comfort with Docker • Ability to work in a Kubernetes environment • Experience with a workflow orchestrator such as Airflow, Prefect, or Dagster • Solid SQL and experience with at least one NoSQL or caching layer, such as Redis, MongoDB, or DynamoDB • Sound computer science fundamentals in data structures and algorithms • Ability to reason about performance and correctness in distributed systems • Strong written communication and ability to work asynchronously across U.S. time zones

🏖️ Benefits

• Fully remote work • Support for learning Scala for candidates strong in Java or Kotlin • Opportunity to work alongside experienced engineers running systems at scale