AI Conversation Research, Evaluation Specialist

🕒 October 22

🌐 United States, Saudi Arabia, +4 more countries – Remote

🇺🇸 United States – Remote 🇸🇦 Saudi Arabia – Remote 🇨🇦 Canada – Remote 🇬🇧 United Kingdom – Remote 🇦🇺 Australia – Remote 🇮🇪 Ireland – Remote

⏰ Full Time

🟡 Mid-level

🟠 Senior

🤖 Artificial Intelligence

👻 Ghost score 12%

What it means: the chance that no one is really hiring for this job.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Gramian Consulting

WebsiteLinkedInAll Job Openings

2 - 10 employees

Founded 2025

💼 Consulting

📦 Logistics

📣 Marketing

Consulting • Logistics • Marketing

Gramian Consulting is a remote-first consulting firm that connects engineering and data/AI talent with organizations through talent augmentation, recruiting, dedicated teams, and contractor management. The firm provides Data & AI services including LLM training and fine-tuning, AI agents and assistants, MLOps, and AI infrastructure, and it offers mentorship and education programs for career readiness, interview preparation, and international market orientation. Rooted in hands-on engineering and recruiting experience, Gramian helps clients scale technical teams and extract business value from AI while developing individual talent.

📋 Description

• Review AI conversations involving real-world interpersonal situations and everyday decision-making • Identify missing context and recommend relevant follow-up questions to improve AI responses • Evaluate whether additional information would materially change an AI-generated recommendation • Distinguish load-bearing information affecting a decision from non-essential or descriptive details • Conduct qualitative reviews and document findings using structured evaluation guidelines • Assess whether conversations consider relevant personal, psychological, social, financial, practical, and organizational factors • Identify missed opportunities for clarification, deeper understanding, and information gathering • Review conversations involving relationships, work, education, family, wellbeing, planning, and resource constraints • Collaborate with research teams to ensure consistent, evidence-based assessments • Maintain accuracy, clarity, and consistency while meeting project deadlines

🎯 Requirements

• At least 3–5 years of relevant direct professional experience in one or more listed domains • Licensed Social Work pathway: Master's degree in Social Work (MSW) or an equivalent graduate-level qualification • Licensed Social Work pathway: LCSW, LICSW, LMSW, or an equivalent applicable social work license or credential • Experience in client assessment, case review, interviewing, counseling, or related social-work practice • Counseling, Therapy, or Psychology pathway: Master's degree in a relevant discipline with an LMFT, LPC, LCPC, or equivalent counseling or therapy license or credential; or PhD or PsyD in Clinical, Counseling, Social, or Industrial-Organizational Psychology • Relevant experience in counseling, therapy, psychological assessment, interviewing, case review, or other client-facing professional work • Qualitative Research, HCI, or Communication pathway: MA, PhD, or equivalent advanced expertise in Human-Computer Interaction, Communication, Anthropology, Sociology, Qualitative Research, or a closely related discipline • Experience conducting qualitative research, interviews, user research, conversation analysis, thematic analysis, or qualitative coding • Ability to analyze conversations systematically and identify missing context, communication patterns, and opportunities for clarification • Mediation, Coaching, Student Services, Organizational Consulting, or Related Advisory Fields pathway: relevant professional qualification, credential, or demonstrated subject-matter expertise • At least 5 years of direct advisory or client-facing experience for candidates qualifying through this pathway • Experience in coaching, mediation, student advising, organizational consulting, or comparable professional work • Demonstrated ability to gather context, understand individual or organizational needs, and provide informed recommendations

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume