Senior Data Engineer

🕒 April 26

🗽 New York – Remote

🗽 New York – Remote

💵 $128.2k - $157k / year

⏰ Full Time

🟠 Senior

🚰 Data Engineer

🦅 H1B Visa Sponsor

This company has sponsored H1B visas in the past.

Apply Now

Find Similar Remote Jobs

Receive Emails For Remote Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Avaya

WebsiteLinkedInAll Job Openings

5001 - 10000 employees

🤝 B2B

💰 Post-IPO Debt on 2022-06

AI • B2B • Communications

Avaya is a leading provider of customer experience and communication solutions, empowering enterprises to innovate without disruption. The company offers a comprehensive suite of AI-driven technologies designed to enhance customer journeys, streamline employee engagement, and drive business growth, ensuring consistent, high-quality interactions across all channels. Avaya's platforms, such as the Avaya Experience Platform and Communication & Collaboration Suite, facilitate seamless cross-channel communication and collaborative workflows for organizations across a variety of industries.

📋 Description

• Design, build, and operate low-latency streaming pipelines (Kafka, Spark Structured Streaming) and robust batch ETL/ELT on Databricks Lakehouse. • Establish reliable orchestration and dependency management (Airflow), with strong SLAs and on-call readiness for business-critical data flows. • Model, optimize, and document curated datasets and interfaces that serve analytics, product features, and AI workloads. • Implement data quality checks, observability, and backfills; drive root-cause analysis and incident prevention. • Partner with application teams (Go/Java), analytics, and ML/AI to ship data products into production. • Build and maintain datasets and services that power RAG pipelines and agentic AI workflows (tool-use/function calling). • When Spark/Databricks isn’t optimal, design and operate custom processors/services in Go to meet strict latency or specialized transformation requirements. • Instrument prompt/response and token usage telemetry to support LLMOps evaluation and cost optimization; provide datasets for labeling and golden sets. • Improve performance and cost (storage/compute), review code, and raise engineering standards.

🎯 Requirements

• 6+ years building production-grade data pipelines at scale (streaming and batch). • Deep proficiency in Python and SQL; strong Spark experience on Databricks (or similar). • Advanced SQL: window functions, CTEs, partitioning/z-ordering, query planning and tuning in lakehouse environments. • Hands-on with Kafka (or equivalent) and an orchestrator (Airflow preferred). • Strong data modeling skills and performance tuning for low latency and high throughput. • Production mindset: SLAs, monitoring, alerting, CI/CD, and on-call participation. • Proficient using AI coding assistants (Cursor, Claude Code) as part of daily development. • Proficiency building data services/processors in Go (or willingness to ramp quickly), and familiarity with alternative frameworks (e.g., Flink/Beam) is a plus.

🏖️ Benefits

• performance-related bonus • benefits