Data Processing Software Developer

Vaga não está no LinkedIn

🕒 Outubro 22, 2025

🇺🇸 Estados Unidos – Remoto (EUA)

⏰ Tempo Integral

🟡 Pleno

🟠 Sênior

🧑‍💻 Engenheiro Full-stack

🦅 Patrocina Visto H1B

Esta empresa já patrocinou vistos H1B no passado.

Candidatar-se

Encontrar Vagas Remotas Similares

Receber Emails de Vagas Remotas

📊 Verifique sua pontuação de currículo para esta vaga

Melhore suas chances de conseguir uma entrevista verificando sua pontuação de currículo antes de se candidatar.

Enviar Currículo

DataPelago

SiteLinkedInTodas as Vagas

51 - 200 funcionários

Fundada em 2022

🤖 Inteligência Artificial

🏢 Corporativo

🤝 B2B

Artificial Intelligence • Enterprise • B2B

A DataPelago é uma empresa que fornece um mecanismo universal de processamento de dados (DataPelago Nucleus) e o DataPelago Accelerator para Spark para acelerar drasticamente cargas de trabalho GenAI e de análise. Sua plataforma aproveita a computação acelerada heterogênea (GPUs, FPGAs e CPUs), integra-se com frameworks de código aberto como Spark, Trino e Gluten/Substrait, e visa oferecer uma performance várias ordens de magnitude superior e uma redução significativa de custos sem necessidade de alterações nas aplicações ou dependência de fornecedores. A DataPelago tem como público-alvo profissionais de dados e IA em empresas, oferecendo guias de implantação, benchmarks e estudos de caso que demonstram grandes economias e processamento mais rápido para conjuntos de dados grandes e estruturados, semiestruturados e não estruturados.

Descrição

• Implement software to perform data processing operations efficiently on large data sets as part of an overall data processing engine. • Work closely with technical leads and other individual developers to develop this software and ensure data is processed accurately, efficiently, and reliably as part of the overall analytics engine. • Investigate and address issues in the software and develop necessary enhancements in an ongoing manner. • Working with cross-functional teams to deliver new algorithms and techniques in the core Datapelago GPU-powered, parallel and distributed execution engine • Developing enterprise-grade, highly reliable code and owning the end-to-end delivery of solutions ensuring smooth production rollouts • Designing, implementing, testing, and maintaining enhancements in different components of our engine. • Developing highly modular components and ensuring clean integration with the entire stack • Define algorithms, mechanisms, procedures, and policies for the required functionality. Document them, and secure review and approval from the technical lead(s) • Continuously analyzing and identifying areas for differentiation and improvements in the execution engine • Identifying bottlenecks in scaling and performance and implementing solutions • Investigate and resolve all bugs before and after the release of the software • Support troubleshooting and resolution of issues, incidents, and problems, working in collaboration with others as needed. • Document findings, resolutions, and identify required enhancements to avoid recurrence • Collaborating with other teams and team members and assisting with code reviews, design reviews, and development process improvements

🎯 Requisitos

• S. EE/CS or equivalent with 5+ years of experience or MS with 3+ years’ experience • 5+ years of experience developing database execution engine components including core database operations - joins, aggregations, sorts, analytic functions like window functions, etc., in an enterprise-class database serving large-scale data processing workloads • Experience developing modern analytic databases and familiarity with vectorized execution, parallel and distributed database algorithms, memory management, etc. • Good understanding of SQL and familiarity with Lakehouse architectures and technologies • Solid experience in developing performant code that minimizes overheads and latency and maximizes throughput • Experienced with the best practices in developing high quality, reliable software • Experience developing, evaluating, and troubleshooting high-performance software • Strong programming ability in C, C++. Rust experiences a strong plus • Strong development experience on Linux platforms