Senior Platform Engineer, AI Evaluation

Job not on LinkedIn

6 hours ago

🏄 California – Remote

Although this job is listed in California, this company may be open to hiring remote in other states.

💵 $137.9k - $172.3k / year

⏰ Full Time

🟠 Senior

🏗️ Platform Engineer

Apply Now

Receive Emails with Similar Jobs

Khan Academy Türkçe

WebsiteLinkedInAll Job Openings

Education

Khan Academy Türkçe is a free educational platform, offering thousands of instructional videos and interactive exercises in Turkish. It serves as both a personal learning resource and a classroom educational tool, covering a wide range of subjects including mathematics, science, computer science, arts, and humanities. The platform caters to all educational levels from elementary to university and emphasizes personalized learning with a vast library that continues to grow as more content is translated into Turkish. Khan Academy Türkçe aims to provide world-class education to anyone, anywhere, without cost, and is utilized by millions of students and teachers across Turkey.

11 - 50 employees

Founded 2012

📚 Education

📋 Description

• Evolve and extend our internal evaluation framework for assessing the quality of our AI-driven experiences at Khan Academy • Work closely with ML data engineers and platform developers to help internal teams adopt an eval-driven development process incorporating offline benchmark tests and online experiments • Gather internal requirements, get buy-in for changes, and develop documentation and training materials • Be fluent in the range of offline and online evaluation strategies, and when to apply the techniques over the lifecycle of development • Have intuitions about how to specify eval pipelines succinctly using declarative syntax • Understand the role of stratified datasets and ground truth labeling • Appreciate the range of eval scoring schemes from human raters to automated LLMs-as-judge

🎯 Requirements

• Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related field • 5 years of Software Engineering experience with 2+ of those years working on the evaluation of generative AI systems • Strong programming skills in Go, Python, SQL, and at least one data pipeline framework (e.g., Airflow, Dagster, Prefect) • Familiarity with the architecture of large language models and their industry-standard APIs • Experience with labeling platforms (e.g., Label Studio, Scale AI, Toloka) and human-in-the-loop concerns such as rubric development and inter-rater agreement • Exposure to MLOps practices such as model registry, feature store, or continuous evaluation • Background in education technology or other human-centered AI applications

🏖️ Benefits

• Competitive salaries • Ample paid time off as needed – Your well-being is a priority • 8 pre-scheduled Wellness Days in 2026 occurring on a Monday or a Friday for a 3-day weekend boost • Remote-first culture - that caters to your time zone, with open flexibility as needed, at times • Generous parental leave • An exceptional team that trusts you and gives you the freedom to do your best • The chance to put your talents towards a deeply meaningful mission and the opportunity to work on high-impact products that are already defining the future of education • Opportunities to connect through affinity, ally, and social groups • And we offer all those other typical benefits as well: 401(k) + 4% matching & comprehensive insurance, including medical, dental, vision, and life