Software Engineer, Big Data

🔥 0 minutes ago

🇺🇸 United States – Remote

💵 $135k - $185k / year

⏰ Full Time

🟢 Junior

🟡 Mid-level

🧑‍💻 Full-stack Engineer

🚫👨‍🎓 No degree required

🦅 H1B Visa Sponsor

This company has sponsored H1B visas in the past.

👻 Ghost score 0%

What it means: the chance that no one is really hiring for this job.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

ZipRecruiter

WebsiteLinkedInAll Job Openings

1001 - 5000 employees

Founded 2010

🎯 Recruiter

👥 HR Tech

🤝 B2B

💰 $550M Post-IPO Debt on 2022-01

Recruitment • HR Tech • B2B

ZipRecruiter is an online employment marketplace that connects job seekers with employers. It offers a platform where candidates can create profiles and search for jobs, while employers can create accounts to list job openings. The platform sends instant job alerts to potential candidates using powerful matching technology, ensuring convenience for job seekers and efficient recruitment for businesses. ZipRecruiter's services are utilized by small & medium businesses, enterprise businesses, and individual job seekers alike, aiming to streamline the job search and recruitment process.

📋 Description

• Design, build, deploy, and maintain data pipelines and systems that make web events available in the datalake and via event streams for downstream applications • Process large, evolving datasets using batch and streaming techniques while balancing speed, efficiency, and reliability • Design, build, deploy, and maintain MLOps frameworks and systems that accelerate ML development across teams and products • Improve the velocity, quality, observability, and reliability of ML development and ML-powered products • Design, build, deploy, and maintain data documentation frameworks and tooling • Design, build, deploy, and maintain Sift, an agentic data exploration and analysis tool • Build scalable, reliable, and efficient infrastructure, tools, and processes that enable engineering teams to deliver products and services • Build and maintain tools for data processing, dataset ingestion and discoverability, data pipeline orchestration, data documentation, data analysis, ML model pipelines, and experiments • Help developers leverage AI through an agentic AI data exploration and analysis tool and an LLM Gateway

🎯 Requirements

• 2+ years of professional software development experience, including building and operating data pipelines that feed into a customer facing product • Hands-on Apache Spark experience: writing, tuning and debugging batch and/or streaming jobs at scale • Experience deploying and operating containerized services and batch jobs • Solid computer science fundamentals: data structures, algorithms, distributed systems, and testing • Professional software development experience, including building and operating data pipelines and infrastructure that feed into a customer facing product • Preferred: 4+ years of professional software development experience with a focus on data-intensive systems • Preferred: Working knowledge of AWS (S3, IAM, EC2, MSK or equivalents) and instrumenting jobs with metrics, logs and alerts • Preferred: Experience with a data lake stack: parquet on object storage, a data catalog such as AWS Glue, and an open table format such as Apache Iceberg or Delta Lake • Preferred: Streaming experience with Kafka producers and consumers, Apache Flink, and/or Spark Structured Streaming • Preferred: Experience with Apache Airflow, including authoring DAGs, backfills, dependency management and on-call for scheduled pipelines • Preferred: Experience with open table format internals: schema evolution, partitioning, and compaction • Preferred: Experience deploying services on Kubernetes with Docker • Preferred: Experience building and maintaining a data platform powering a user-facing product, including streaming and batch data ingestion, ETL pipelines, data permissions management, and data discovery • BS/MS/PhD in Computer Science, Mathematics, Physics or a related field, or equivalent practical experience (preferred) • Must be able to work in the United States, as indicated by the US base salary range and US-based positions language

🏖️ Benefits

• Competitive compensation • Exceptional benefits package • Flexible Vacation & Paid Time Off • Employer-matched 401(k) plan • Equity, bonuses, commission, or other forms of compensation may also be provided depending on the position offered • Medical, financial, and other benefits

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume