AI Evaluation Specialist

Job not on LinkedIn

🕒 July 9

🇺🇸 United States – Remote

💵 $70 / hour

⏳ Contract/Temporary

🟡 Mid-level

🟠 Senior

🤖 Artificial Intelligence

👻 Ghost score 24%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Weekday (YC W21)

WebsiteLinkedInAll Job Openings

11 - 50 employees

Founded 2021

💼 Consulting

👥 HR Tech

☁️ SaaS

Consulting • HR Tech • SaaS

Weekday is a modern recruitment platform that combines AI technologies with a vast database of potential candidates, aiming to streamline the hiring process for companies in India. They offer various services, including a proactive outreach approach that helps employers connect with top talent, as well as tools for candidates to easily apply for jobs. Weekday's emphasis on candidate engagement through multiple channels, including email, WhatsApp, and phone calls, sets it apart in the competitive landscape of recruitment agencies.

📋 Description

• Review AI-generated content for accuracy, logical reasoning, completeness, and clarity. • Identify factual errors, reasoning gaps, inconsistencies, and unsupported conclusions. • Assess responses using structured evaluation frameworks and detailed quality guidelines. • Write clear, concise, and evidence-based rationales explaining evaluation decisions. • Highlight both strengths and areas for improvement in AI-generated outputs. • Apply consistent judgment across a wide range of evaluation tasks. • Follow detailed project instructions and standardized assessment criteria. • Ensure evaluations are objective, accurate, and reproducible. • Complete assignments independently while maintaining high quality standards.

🎯 Requirements

• Bachelor's degree from a globally recognized university (top-ranked institutions preferred). • Excellent analytical thinking and problem-solving abilities. • Strong written communication skills with the ability to explain complex reasoning clearly and precisely. • Exceptional critical reading skills, including the ability to identify: • Nuanced arguments • Implicit meaning • Logical inconsistencies • Missing context • Weak or unsupported reasoning • Strong attention to detail and ability to consistently apply structured evaluation guidelines. • Ability to work independently and manage assigned tasks efficiently. • Native-level English fluency. • Experience in content evaluation, research, quality assurance, editing, or analytical review (preferred). • Familiarity with artificial intelligence, large language models, or AI evaluation methodologies (preferred). • Experience working with structured annotation or assessment frameworks (preferred). • Ability to produce thoughtful, objective, and well-supported written evaluations under defined quality standards (preferred).