AI Rater Guidelines Writer, Linguist, Instructional Designer

Job not on LinkedIn

🕒 July 29

🇺🇸 United States – Remote

💵 $45 - $65 / hour

⏳ Contract/Temporary

🟡 Mid-level

🟠 Senior

📚 Learning and Development

👻 Ghost score 24%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Weekday (YC W21)

WebsiteLinkedInAll Job Openings

11 - 50 employees

Founded 2021

💼 Consulting

👥 HR Tech

☁️ SaaS

Consulting • HR Tech • SaaS

Weekday is a modern recruitment platform that combines AI technologies with a vast database of potential candidates, aiming to streamline the hiring process for companies in India. They offer various services, including a proactive outreach approach that helps employers connect with top talent, as well as tools for candidates to easily apply for jobs. Weekday's emphasis on candidate engagement through multiple channels, including email, WhatsApp, and phone calls, sets it apart in the competitive landscape of recruitment agencies.

📋 Description

• Translate complex and ambiguous program requirements into clear, concise, and easy-to-follow rater guidelines. • Design comprehensive evaluation rubrics and scoring frameworks that enable consistent decision-making across a variety of AI tasks. • Review existing documentation to identify ambiguity, inconsistencies, contradictions, and coverage gaps, then revise guidelines to improve clarity and usability. • Convert subject-matter requirements from domains such as finance, retail, insurance, legal, and sports into structured evaluation instructions that non-domain raters can apply confidently. • Collaborate with research teams, program managers, and subject matter experts to maintain consistency across multiple guideline sets. • Develop documentation that addresses edge cases, exceptions, and complex evaluation scenarios while minimizing reviewer escalation.

🎯 Requirements

• Minimum 3 years of professional experience in Linguistics, Instructional Design, Technical Writing, AI Content Development, or a closely related field. • Direct experience creating, refining, or maintaining evaluation guidelines, rating rubrics, or reviewer instructions within Generative AI, RLHF, human evaluation, or AI data annotation environments. • Demonstrated ability to work across multiple subject areas and convert domain-specific knowledge into clear, structured documentation. • Proven experience resolving ambiguity and improving written specifications with measurable before-and-after improvements. • Strong analytical thinking with exceptional attention to detail. • Demonstrated career progression and professional growth. • Ability to commit reliably to 35+ hours per week during weekdays. • Outstanding written communication skills with the ability to explain nuanced concepts in a precise, consistent, and easy-to-understand manner.