Content Specialist III – AI Evaluation, Prompting

Job not on LinkedIn

🔥 2 hours ago

🇺🇸 United States – Remote

⏳ Contract/Temporary

🟡 Mid-level

🟠 Senior

✍️ Content Marketing Manager

👻 Ghost score 13%

What it means: the chance that no one is really hiring for this job.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Linda Werner & Associates, Inc

WebsiteLinkedInAll Job Openings

51 - 200 employees

Founded 1997

💼 Consulting

📦 Logistics

📣 Marketing

Consulting • Logistics • Marketing

Linda Werner & Associates, Inc. is a woman-owned technical consulting and staffing firm with over 25 years of experience specializing in placing skilled professionals in technical communication, project management, development, and related consulting roles. The company focuses on personalized, relationship-driven recruitment and consultant placement for client projects, emphasizing cultural fit, long-term relationships, and candidate fulfillment.

📋 Description

• Test new AI model versions across a variety of topics, use cases, and conversation types • Evaluate model responses against established rubrics, guidelines, and quality standards • Identify and document specific examples of successful and unsuccessful model behavior • Write and refine system prompts that help shape model personality, tone, and behavior • Assess whether evaluation rubrics and quality standards effectively measure model performance • Perform quality reviews of both human and agent-based evaluations to ensure accuracy and compliance • Conduct hands-on experiments with AI models and products • Investigate model failures and document findings clearly • Apply detailed instructions and evaluation criteria consistently across a high volume of work • Adapt quickly as product priorities, model behavior, and evaluation needs evolve • Balance quality, speed, volume, and independent judgment while following detailed instructions • Translate observations about AI responses into clear and actionable feedback

🎯 Requirements

• 5+ years of experience in writing, editing, journalism, production, linguistics, STEM, coding, policy, or another relevant subject matter field • 1+ year of hands-on AI experience preferred, including prompting, annotation, evaluation, or red teaming • Bachelor’s degree or equivalent experience • Deep expertise in at least one subject area with the ability to evaluate content as a subject matter expert • Strong judgment and attention to detail with the ability to consistently apply detailed rubrics, verify information, and identify quality issues • Experience evaluating generative AI or large language model outputs • Strong fact-checking and research skills • Ability to verify claims against original sources • Experience conducting detailed failure investigations • Clear written communication and documentation skills • Ability to flag questions and blockers early • Comfortable shifting priorities as product needs change • Ability to work 40 hours per week, Monday to Friday, on an 8-hour shift • English language proficiency required • Must address current or future US work sponsorship requirements

🏖️ Benefits

• Dental insurance • Health insurance • Health savings account • Life insurance • Paid time off • Retirement plan • Vision insurance • 8 hour shift • Monday to Friday

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume