Senior Director, Software Engineering

🔥 15 hours ago

🌽 Illinois – Remote

🌽 Illinois – Remote

💵 $120k - $165k / year

⏰ Full Time

🟠 Senior

🧑‍💻 Full-stack Engineer

👻 Ghost score 0%

What it means: the chance that no one is really hiring for this job.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Circana

WebsiteLinkedInAll Job Openings

5001 - 10000 employees

🤝 B2B

💼 Consulting

B2B • Consulting

Circana is a market research company that specializes in measuring and accelerating demand for brands. They utilize big data and an AI data-visualization platform to tackle complex business challenges, guiding clients through innovation and meeting consumer demand. With over a century of combined experience from their legacy as IRI and NPD, Circana serves more than 7,000 global brands and retailers, providing insights into both online and in-store purchasing behavior. Their expertise is recognized by prominent media outlets including Bloomberg, CNBC, and The New York Times.

📋 Description

• Design, develop, and optimize scalable data workflows using Python, PySpark, and Airflow • Implement real-time and batch data processing using Spark • Enforce data quality, governance, and security best practices throughout the data lifecycle • Monitor and automate data systems to ensure availability, reliability, and performance • Manage cloud infrastructure and optimize data-processing costs • Implement CI/CD pipelines for reliable data-workflow deployments • Build and optimize large-scale data-processing pipelines using Apache Spark and PySpark • Implement data partitioning, caching, and performance tuning for Spark workloads • Work with structured and unstructured data to support analytics and machine-learning initiatives • Design and maintain Airflow DAGs to automate complex data workflows • Monitor, troubleshoot, and optimize job execution and dependencies • Lead a team of data engineers and provide technical guidance and mentorship • Promote coding standards, version control, and documentation best practices • Engage directly with clients to understand business needs and communicate technical concepts • Collaborate with engineering, product, and architecture peers to drive scalable, maintainable products • Partner with Program Management to align delivery timelines and mitigate risks

🎯 Requirements

• 8+ years of experience in data engineering • Expertise in Python, PySpark, Apache Spark, and Apache Airflow • Strong programming skills in Python and SQL • Ability to write efficient and maintainable code • Deep understanding of Spark internals, including RDDs, DataFrames, DAG execution, and partitioning • Experience with Airflow DAGs, scheduling, and dependency management • Knowledge of Git, Docker, Kubernetes, Terraform, and DevOps best practices for CI/CD workflows • Experience optimizing large-scale data processing • Experience leading teams and working in Agile/Scrum environments • Experience with common data warehouse and/or data lake platforms • Knowledge of cloud technologies, including Azure, AWS, or Google Cloud • Proven experience using generative AI tools for software development, data engineering, system design, testing, and troubleshooting • Strong understanding of AI-assisted engineering practices, including prompt engineering, code generation, automated testing, and AI-driven productivity workflows • Ability to identify opportunities to apply AI solutions to improve productivity, code quality, and operational efficiency • Strong communication and collaboration skills • Experience with ML Engineering is a plus

🏖️ Benefits

• Flexible and accommodating work environment • Paid time off • Medical insurance • Dental insurance • Vision insurance • 401(k) • Bonus pay eligibility • Accessibility accommodations for applicants with disabilities • Background checks conducted in accordance with local legislation • Current employer will not be contacted without permission

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume