Staff Agentic Data Engineer – Data Platform

Job not on LinkedIn

🔥 1 minute ago

🇺🇸 United States – Remote

⏰ Full Time

🔴 Lead

🚰 Data Engineer

👻 Ghost score 10%

What it means: the chance that no one is really hiring for this job.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Nucleus Security

WebsiteLinkedInAll Job Openings

51 - 200 employees

Founded 2017

🔒 Cybersecurity

📋 Compliance

🏢 Enterprise

💰 $13M Series B on 2024-02

Cybersecurity • Compliance • Enterprise

Nucleus Security is a comprehensive platform designed to scale and automate vulnerability and exposure management programs. It offers an extensive ecosystem with over 150 integrations, allowing for robust vulnerability aggregation, risk prioritization, and remediation automation. The platform provides valuable solutions for both public sector entities, including federal and state governments, as well as private sectors by centralizing security and simplifying compliance processes. Nucleus Security is recognized as a leader in continuous vulnerability management, offering the capability to unify asset, vulnerability, and threat data to effectively mitigate critical exposures. With features like plan of action and milestones automation, the platform is acclaimed for enhancing risk-based vulnerability management and application security, making it a top choice among enterprise users for its scalability, automation, and ability to achieve compliance with standards like FedRAMP and SOC2.

📋 Description

• Own the architecture and evolution of large-scale data processing and analytics systems, from ingestion and normalization through transformation, aggregation, and customer-facing analytics • Design and evolve data models supporting high-volume analytical workloads while balancing query performance, storage efficiency, maintainability, and scalability • Design and optimize complex analytical SQL queries across large datasets, identifying performance bottlenecks and developing scalable solutions • Architect distributed and asynchronous job processing systems for reliable processing of large data volumes across concurrent workloads • Design and evolve Spark, Flink, or similar data-processing workflows for large-scale transformation, enrichment, aggregation, and computation • Drive effective use of SingleStore/MemSQL, including schema design, indexing and partitioning strategies, query optimization, and workload design • Establish architectural patterns for batch processing, data transformation, job orchestration, failure recovery, idempotency, and reprocessing • Diagnose complex performance and scalability issues across databases, distributed processing systems, application services, and infrastructure • Partner with engineering and product teams to translate ambiguous analytical and data-processing requirements into scalable technical architectures • Define technical standards and reusable patterns for data modeling, data processing, and analytical query development • Provide technical leadership across engineering initiatives, influencing architecture and mentoring engineers while remaining hands-on with implementation • Identify architectural limitations and lead improvements to increase platform scalability, reliability, and performance

🎯 Requirements

• 10+ years of professional software engineering, data engineering, or data platform experience, with significant experience designing and operating production data systems • Deep expertise in relational and analytical data modeling, including designing schemas for large-scale analytical and reporting workloads • Expert-level SQL, with demonstrated experience designing, analyzing, and optimizing complex analytical queries over large datasets • Strong experience with SingleStore/MemSQL or comparable distributed SQL/analytical database technologies • Significant hands-on experience with Spark, Flink or similar technologies, including designing and optimizing distributed data-processing workloads • Strong understanding of distributed data processing, including partitioning, parallelism, data skew, resource utilization, and performance optimization • Experience designing high-throughput job processing systems, including scheduling, concurrency management, retries, failure recovery, idempotency, and observability • Experience designing data pipelines that support batch ingestion, transformation, enrichment, aggregation, and downstream analytics • Strong understanding of database performance characteristics, including indexing, partitioning, query execution plans, data distribution, and memory/compute trade-offs • Experience identifying and resolving performance bottlenecks across application, database, and data-processing layers • Proficiency in Python, Java or another production programming language used to build data-processing and distributed systems • Strong understanding of testing strategies for data-intensive systems, including unit, integration, data-quality, and end-to-end testing • Proven ability to independently own technically ambiguous problems and drive them from investigation through production implementation • Demonstrated ability to influence architecture and technical direction across multiple teams without relying solely on organizational authority • Ability to communicate complex data architecture and performance trade-offs clearly to engineers, product stakeholders, and technical leadership • Ability to collaborate effectively with distributed teams across multiple time zones • Preferred: Deep production experience with SingleStore/MemSQL, particularly with large-scale analytical workloads • Preferred: Advanced experience with pipeline performance tuning, partitioning strategies, memory management, and optimization of large distributed workloads • Preferred: Experience designing platforms that support a combination of high-volume ingestion and low-latency analytical queries • Preferred: Experience building or operating distributed job execution and orchestration systems • Preferred: Experience with large-scale cybersecurity, vulnerability, asset, telemetry, or event datasets • Preferred: Experience working with vulnerability scanning technologies across the security stack, including SCA, SAST, DAST, IAST, vulnerability scanning, and container security • Preferred: Experience designing systems for incremental processing, backfills, replay/reprocessing, deduplication, and late-arriving data • Preferred: Experience with data observability and operational monitoring for large-scale processing workloads • Preferred: Experience integrating AI/LLM-driven capabilities with data platforms, including preparing and processing large datasets for AI-assisted analytical workflows • Preferred: Experience operating data-intensive applications on Linux and cloud infrastructure • Preferred: Familiarity with Agile/Scrum methodologies in a professional engineering environment

🏖️ Benefits

• Flexible PTO • 401k company contribution (not a match, a contribution) • Generous education and training budget • 100% company-paid healthcare • Fully remote work

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume