AI Quality Analyst, LLM – Personalization – Chinese

Job not on LinkedIn

🔥 19 minutes ago

💵 $11 - $14 / hour

⏳ Contract/Temporary

🟡 Mid-level

🟠 Senior

🤖 Artificial Intelligence

🗣️🇨🇳 Chinese Required

Where this job comes from

Posted on Careerflow.ai's careers page ↗Careerflow.ai's careers page, not through a job board

First seen by Remote Rocketship 19 minutes ago

Apply Now
Logo of Careerflow.ai

Careerflow.ai

11 - 50 employees

Founded 2022

🤖 Artificial Intelligence

👥 HR Tech

☁️ SaaS

💰 $700k PRE-SEED on 2024-05

Artificial Intelligence • HR Tech • SaaS

Careerflow. ai is an AI-powered job search platform that helps users create and optimize resumes and cover letters, track applications, improve LinkedIn profiles, and prepare for interviews. It also provides career tools to organizations such as university career centers, bootcamps, and outplacement providers.

📋 Description

• Evaluate a new Gemini personalization feature using information from past Gemini conversations, Gmail, Google Search, and YouTube activity • Design and execute multi-turn conversational prompts, typically 1–5 turns, using personal information and experiences • Evaluate model responses against the intent of the starting prompt • Analyze responses for Grounding issues, unsupported claims, flawed inferences, and hallucinations • Assess Integration quality and whether personal data is used naturally without robotic overnarrating • Stack-rank two model responses side-by-side based on helpfulness, ease of use, and enjoyability • Write clear, defensible rationales referencing specific conversation turns • Extract and verify Debug Info to confirm chat summaries and data sources were properly utilized • Delete evaluation conversations to maintain data hygiene and prevent pollution of future chat history

🎯 Requirements

• Ability to read and write in Chinese with a high degree of competence • Full-time availability in the local time zone • Ability to evaluate nuanced and ambiguous AI responses, specifically personalization quality • Experience designing creative, multi-turn prompts based on personal context • Understanding of personalization concepts, including identifying incorrect personalization, poor inferences, and forced connections • Ability to review Side-by-Side (SxS) model responses and identify subtle differences in naturalness and overnarrating • Superior ability to write clear, concise, and structured rationales explicitly referencing specific turn numbers • Ability to provide constructive feedback and detailed annotations • Excellent communication and collaboration skills • Ability to work independently in a remote setting • Desktop/laptop with a good internet connection • BS/BA degree or equivalent experience in a relevant field • Experience in data annotation, AI quality evaluation, content moderation, or a related role is strongly preferred • Willingness to use a primary personal Google account and enable personal data sources • Minimum commitment of 30 hours per week, with at least 4 hours per day and 4 hours of overlap with PST

🏖️ Benefits

• $21 (USD) per hour for US-based candidates • Flexible schedule options of 30 or 40 hours per week

Apply Now

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Similar Jobs

🕒 November 5

RWS Group

5001 - 10000

💼 Consulting

🏥 Healthcare

⚖️ Legal

Freelance voice contributor recording Chinese Simplified speech data for RWS’s AI evaluation project. Producing accurate, natural recordings to improve speech and language technologies.

🗣️🇨🇳 Chinese Required

🕒 November 5

Invisible Technologies

201 - 500

💼 Consulting

🏥 Healthcare

📦 Logistics

Malay voice actor recording scripted speech and evaluating AI-generated outputs for Meridial’s voice models. Providing linguistic feedback to improve pronunciation, tone, pacing, and emotional expression.

🇲🇾 Malaysia – Remote

💵 $8 - $65 / hour

💰 $100M Series unknown on 2025-10

⏳ Contract/Temporary

🟡 Mid-level

🟠 Senior

🤖 Artificial Intelligence

🗣️🇲🇾 Malay Required

🕒 November 5

Invisible Technologies

201 - 500

💼 Consulting

🏥 Healthcare

📦 Logistics

Malay language specialist challenging and evaluating AI language models. Documenting linguistic errors and improving prompt engineering and evaluation metrics for Meridial.

🇲🇾 Malaysia – Remote

💵 $6 - $65 / hour

💰 $100M Series unknown on 2025-10

⏳ Contract/Temporary

🟡 Mid-level

🟠 Senior

🤖 Artificial Intelligence

🗣️🇲🇾 Malay Required

🕒 November 4

Invisible Technologies

201 - 500

💼 Consulting

🏥 Healthcare

📦 Logistics

Freelance Indonesian language specialist challenging language models on grammar, semantics, and dialects. Documenting errors and improving prompt engineering and evaluation metrics for Meridial.

🇲🇾 Malaysia – Remote

💵 $6 - $65 / hour

💰 $100M Series unknown on 2025-10

⏳ Contract/Temporary

🟡 Mid-level

🟠 Senior

🤖 Artificial Intelligence

🗣️🇮🇩 Indonesian Required

🕒 November 1

Invisible Technologies

201 - 500

💼 Consulting

🏥 Healthcare

📦 Logistics

Chinese language specialist training Meridial’s AI models through linguistic evaluation. Testing language reasoning, documenting failures, and improving prompts and evaluation metrics.

🇲🇾 Malaysia – Remote

💵 $6 - $65 / hour

💰 $100M Series unknown on 2025-10

⏳ Contract/Temporary

🟡 Mid-level

🟠 Senior

🤖 Artificial Intelligence

🗣️🇨🇳 Chinese Required

Built by Lior Neu-ner. I'd love to hear your feedback — Get in touch via DM or support@remoterocketship.com
Follow Remote Rocketship on LinkedInFollow Remote Rocketship on InstagramFollow Remote Rocketship on FacebookFollow Remote Rocketship on TikTok