Generative AI Post-Doctorate Fellow

Job not on LinkedIn

October 24

Apply Now

Receive Emails with Similar Jobs

The College Board

WebsiteLinkedInAll Job Openings

Education • Non-profit

The College Board is a mission-driven, not-for-profit organization dedicated to promoting excellence and equity in education. It connects students to college success and opportunity by supporting them in building essential skills for higher education and removing barriers to access. The College Board operates with a focus on respect, diversity, inclusion, and collaboration, with a diverse and dedicated team of professionals. Based in several locations across the United States, they strive to modernize operations and increase their digital footprint, supporting students and families nationwide.

1001 - 5000 employees

Founded 1900

📚 Education

🤝 Non-profit

📋 Description

• Develop best practices and design principles for new AI-driven learning & assessment prototypes (33%) • Survey and synthesize the research base on adaptive learning, gamification, and non-cognitive assessment to define evidence-backed design heuristics. • Lead co-design sprints with psychometricians, UX researchers, and engineers to embed those heuristics in prototypes and pilots. • Build evaluation frameworks (validity, reliability, engagement, fairness) to judge prototype performance and iterate rapidly. • Author “Design Principles” playbooks and contribute to internal standards that guide future College Board AI initiatives. • Distill learning-science and assessment best practices into effective LLM contexts (33%) • Translate analytic rubrics, exemplar responses, and scoring anchors into high-leverage prompt libraries, system messages, and retrieval chains for LLM-based tutors and scorers. • Prototype few-shot and chain-of-thought prompting strategies that surface metacognitive feedback and scaffold student reasoning. • Collaborate with engineers to automate rubric-to-prompt conversions and ensure version control across iterative assessment designs. • Establish validation pipelines—quantitative (e.g., QWK, IRT metrics) and qualitative (e.g., think-aloud reviews)—to monitor alignment between LLM outputs and intended learning goals. • Supporting LLM prototype experiments informed by learning science/assessment best practices (33%) • Co-design and analyze rapid LLM prototype experiments (A/B & quasi-experimental) to validate pedagogical soundness, fairness, and psychometric quality.

🎯 Requirements

• PhD (completed by Aug 2025) in Learning Sciences, Educational Psychology, AIED, Measurement, Cognitive Science, or related field • Demonstrated research on adaptive/gamified or non-cognitive assessment design • Hands-on experience with large-language-model prompting / evaluation (OpenAI, Anthropic, etc.) or learning-analytics pipelines. • Strong expertise in statistical modeling and data analysis using appropriate statistical methods and software (e.g., SAS, Stata, SPSS, R, Python); familiarity with IRT or ML fairness a plus. • Experience with large, national datasets related to the education industry and the ability to work accurately and quickly with new datasets • Experience using SQL to access data for data analysis and/or a willingness to learn • Experience with survey design and analysis, and Qualtrics a plus • Adept problem-solving skills, including using data to inform decisions and actions • The ability to travel 4-6 times a year to College Board offices or on behalf of College Board business • Authorization to work in the United States

🏖️ Benefits

• Annual bonuses and opportunities for merit-based raises and promotions • A mission-driven workplace where your impact matters • A team that invests in your development and success