Customer Support Specialist, AI Response Evaluation

Job not on LinkedIn

🕒 March 28

☕ Washington – Remote

☕ Washington – Remote

💵 $30 - $40 / hour

⏳ Contract/Temporary

🟡 Mid-level

🟠 Senior

💝 Customer Support

Apply Now

Find Similar Remote Jobs

Receive Emails For Remote Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

mpathic

WebsiteLinkedInAll Job Openings

11 - 50 employees

Founded 2021

🤖 Artificial Intelligence

🔌 API

☁️ SaaS

Artificial Intelligence • API • SaaS

mpathic is an AI-driven safety and analytics company that offers SaaS products and APIs to record, transcribe, analyze, and improve human–AI and human–human interactions in clinical, life‑science, and enterprise settings. Their offerings include speech-to-text transcription, PII redaction, behavior and checklist analytics, human-curated benchmarking datasets, red‑teaming expertise, and an Observing Agent API to add domain-informed guardrails for healthcare and medical agents. mpathic targets life sciences, clinical research organizations, healthcare providers, insurers, and HR/enterprise teams focused on empathy, safety, bias reduction, and compliance.

📋 Description

• Reviewing AI-generated customer support responses for accuracy, clarity, and helpfulness • Evaluating tone, empathy, and professionalism in customer interactions • Identifying poor support behaviors (e.g., dismissiveness, lack of resolution, incorrect guidance) • Assessing whether AI responses appropriately address customer intent and resolve issues • Evaluating how AI handles frustration, confusion, or escalating user behavior • Identifying gaps in responses, including missing steps, unclear instructions, or lack of follow-through • Detecting overconfident, misleading, or incorrect responses • Evaluating multi-turn conversations for consistency, escalation handling, and resolution quality • Identifying patterns of agreement without problem-solving (e.g., empty empathy or unhelpful reassurance) • Participating in or reviewing red teaming exercises to surface failure modes in support scenarios • Supporting quality assurance (QA) to ensure consistency across evaluations • Documenting edge cases, failure patterns, and customer experience risks • Providing structured written feedback to internal teams • Collaborating with interdisciplinary teams on AI quality, safety, and evaluation frameworks • Maintaining strict confidentiality and quality standards.

🎯 Requirements

• Successful candidates are detail-oriented, customer-focused, and experienced in evaluating or delivering high-quality support in fast-paced environments. • Professional experience in one or more of the following: Customer support (chat, email, or phone) • Call centers or contact centers • Technical support or help desk roles • Online support environments (SaaS, e-commerce, fintech, etc.) • Strong understanding of high-quality customer experience and service standards • Effective communication in customer-facing interactions • Common support workflows (triaging, troubleshooting, escalation) • How customers interpret and respond to support interactions • Ability to identify: • Poor or incomplete resolutions • Incorrect or misleading information • Inappropriate tone (e.g., robotic, dismissive, overly casual) • Lack of empathy or ineffective de-escalation • Failure to follow support best practices • Strong written communication skills and ability to clearly explain reasoning. • Experience with or interest in: • Evaluating conversations or QA for support teams • Working with AI tools, chatbots, or automated systems • Assessing how systems perform under real customer behavior. • Comfort with: • Tech tools and platforms (Slack, spreadsheets, dashboards) • Evaluating AI-generated responses (no coding required, but must be tech-comfortable) • Ambiguity, iteration, and feedback-driven workflows. • Willingness to: • Sign NDAs and work with sensitive content. • Nice to Have (Not Required) • Experience in QA, coaching, or training within support teams • Background in trust & safety or content moderation • Experience with chatbot or AI-assisted support tools • Familiarity with CX metrics (CSAT, QA scoring, resolution rate) • Interest in AI, automation, or improving customer experience systems.