Senior Data Engineer

🔥 0 minutes ago

🐊 Florida, New Jersey, +3 more states – Remote

🐊 Florida – Remote 🏖️ New Jersey – Remote 🗽 New York – Remote 🌲 North Carolina – Remote ⚔️ Virginia – Remote

💵 $140k - $160k / year

⏰ Full Time

🟠 Senior

🚰 Data Engineer

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Hack The Box

WebsiteLinkedInAll Job Openings

51 - 200 employees

Founded 2017

🔒 Cybersecurity

📚 Education

🎮 Gaming

Cybersecurity • Education • Gaming

Hack The Box is a cutting-edge online platform that provides cybersecurity education and training through gamified experiences. It offers a wide range of services including hands-on labs, cybersecurity competitions, and industry-recognized certifications, catering to individuals, professional teams, schools, and enterprises. The platform is designed to help users develop and enhance their skills in offensive and defensive security, equipping them to tackle real-world cyber threats. Hack The Box is widely used for team assessments and skill benchmarking across various industries, and hosts engaging events such as capture-the-flag competitions to facilitate interactive learning.

📋 Description

• You will own and evolve our data pipelines on GCP — building new ones, hardening existing ones, improving data quality, and making clean, trustworthy data available across the organisation. • Designing ELT/ETL processes on BigQuery and ClickHouse, building real-time pipelines on Pub/Sub and Kafka with Dataflow (and where it fits, Flink/Spark). • Orchestrating workflows with Airflow, and ensuring data is properly cleaned, modelled, and served for analytics, ML training, and online inference. • Partnering with ML engineers on feature pipelines, monitoring data drift, and keeping models well-fed and retrained as needed. • Consuming and building REST APIs, integrate with third-party SaaS sources, and treating infrastructure as code. • Collaborating closely with Infrastructure, Software Engineering, Product, and ML/AI engineers. • Helping drive the migration off Snowflake onto GCP-native stack — and retire shadow pipelines along the way. • Owning the orchestration layer in Airflow, including SLAs, retries, and data quality gates. • Modeling data for analytics and for ML — including feature pipelines that serve both training and low-latency online inference. • Capturing requirements from stakeholders and translating them into pragmatic, well-scoped data products. • Continuously improving data quality, reliability, observability, and cost efficiency. • Identifying new data sources worth acquiring and integrating them cleanly.

🎯 Requirements

• Strong data modelling and warehouse architecture skills (dimensional modelling, event-driven, lakehouse patterns) • Hands-on experience with GCP data services — BigQuery is a must; Pub/Sub, Dataflow, Bigtable, Cloud Composer are strong pluses • Production experience with streaming pipelines on Dataflow/Beam, Flink, or Spark Structured Streaming, ingesting from Kafka and/or Pub/Sub • Solid SQL and strong Python — you write production-quality code, not just notebooks • Experience with ClickHouse or another columnar OLAP engine in production • Workflow orchestration experience with Airflow (or Prefect/Dagster) • Comfortable with dbt or equivalent transformation frameworks • Experience migrating off legacy warehouses (Snowflake, Redshift, Synapse) onto cloud-native stacks is a plus • Working knowledge of ML in production — feature engineering, feature stores, model deployment, drift monitoring, retraining • Docker & Kubernetes experience • CI/CD mindset, infrastructure-as-code sensibility, and a bias for simple, observable systems • Bonus: CDC tooling (Datastream, Debezium), Vertex AI / Feature Store

🏖️ Benefits

• Medical, Dental & Vision (employee coverage 100% paid for by Hack The Box) • 401K w/ employer match • Employer-paid Life and AD&D Insurance • Supplemental Life Insurance • Short-term and Long-term Disability • Healthcare and Dependent Care FSA • Paid parental leave • 25 annual leave days • Home Office Allowance • Dedicated budget for training and professional development, participation in conferences • State-of-the-art equipment • Full access to the Hack The Box lab offerings; so you can learn how to hack 😉