AI Response Labeler – Annotator

Job not on LinkedIn

🔥 0 minutes ago

💃 Latin America – Remote

💵 $2.2k - $2.5k / month

⏰ Full Time

🟡 Mid-level

🟠 Senior

🤖 Artificial Intelligence

👻 Ghost score 0%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

BPCS, Comprehensive marketing solutions, ltd.

WebsiteLinkedInAll Job Openings

1 - 10 employees

💼 Consulting

📣 Marketing

Consulting • Marketing

BPCS, Comprehensive marketing solutions, ltd. is a marketing agency founded in 2015 by four friends and experienced advertising professionals. The company offers integrated marketing, business consulting, advertising, and marketing strategies. Positioned as a 'big little agency,' BPCS combines the resources of larger firms with the personal touch and commitment of smaller agencies, providing responsive and dedicated services to meet complex marketing challenges.

📋 Description

• Perform side-by-side comparisons of AI-generated responses and select the stronger response using established evaluation criteria. • Assess responses for factual accuracy, relevance, completeness, reasoning, instruction following, clarity, safety, tone, and usefulness. • Evaluate questions and answers, web-search results, file-based and image-based responses, content generation, and single-turn or multi-turn conversations. • Identify unsupported claims, missed instructions, weak reasoning, and incomplete answers. • Apply detailed, scenario-specific guidelines and make decisions when examples do not provide an obvious answer. • Write concise, evidence-based explanations for evaluation decisions when required. • Meet productivity expectations while maintaining accuracy and consistent judgment. • Participate in training, guided practice, calibration, qualification reviews, and ongoing quality reviews. • Incorporate feedback as evaluation guidelines and quality standards evolve. • Generally complete a minimum of 25 evaluation tasks per day, with most tasks taking approximately 15 minutes.

🎯 Requirements

• Excellent written English comprehension and communication skills, including the ability to read complex prompts and guidelines and explain evaluation decisions clearly. • Strong critical-thinking skills and the ability to assess content across a wide range of topics. • Sound judgment when evaluating factuality, reasoning, user intent, and subtle differences in response quality. • Excellent attention to detail and the ability to apply structured criteria consistently. • Comfort with repetitive, focused work and a high volume of evaluations. • Ability to work independently, respond to feedback, and stay aligned with shared quality standards. • During the approximately 30-day training and qualification period, employees must work from 9:00 a.m. to 5:00 p.m. Pacific Time. • All new hires must successfully complete a structured onboarding and qualification program before beginning production work. • Preferred: experience evaluating, ranking, or comparing AI-generated responses. • Preferred: experience with data annotation, content quality assessment, search relevance evaluation, or model-quality review. • Preferred: experience with detailed rubrics, annotation guidelines, or quality benchmarks. • Preferred: experience writing clear rationales supporting evaluation decisions.

🏖️ Benefits

• Medical, dental, and vision coverage • Flexible Spending Account (FSA) • 401(k) retirement plan • Competitive paid time off • Parental leave • Professional growth and development opportunities