AI Operations Analyst

🔥 18 hours ago

🇺🇸 United States – Remote

💵 $105k - $165k / year

⏰ Full Time

🟡 Mid-level

🟠 Senior

⚙️ Operations

👻 Ghost score 0%

What it means: the chance that no one is really hiring for this job.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

FICO

WebsiteLinkedInAll Job Openings

1001 - 5000 employees

Founded 1956

💼 Consulting

🛡️ Insurance

🏥 Healthcare

Consulting • Insurance • Healthcare

FICO is a leading analytics and software company renowned for its FICO® Score, a tool widely used by lenders to assess credit risk. The company offers a comprehensive platform that leverages data, AI, and machine learning to power intelligent decision-making and customer engagement across various industries. FICO's solutions span fraud detection, credit scoring, and customer lifecycle management, making it vital to sectors such as finance and telecommunications. Its innovative products help businesses optimize outcomes through real-time analytics, business composability, and scenario management.

📋 Description

• Review risk-weighted samples of AI chat conversations and Personalized Credit Insights summaries • Flag inaccurate, unsupported, off-topic, or unsafe responses • Categorize member feedback, guardrail blocks, and escalations into a failure-mode taxonomy • Track failure-mode changes over time • Own the golden regression dataset by adding labeled examples from production misses • Maintain consistent labels and document the labeling rubric • Label data to measure and improve automated evaluators • Double-label a share of items to measure reviewer agreement • Publish weekly quality reports covering intent coverage, resolution and containment, refusal and guardrail rates, safety flags, top failure modes, and recommended fixes • Escalate flagged content to Legal, Compliance, and Trust & Safety with severity ratings • Follow escalated items through resolution • Use SQL and dashboards to analyze member questions, experience gaps, and quality drift • Partner with Product, Applied AI Engineering, and QA engineering on review tooling, rubrics, and sampling methods • Verify fixes after each release

🎯 Requirements

• Bachelor's degree in a related field, or equivalent experience • Demonstrated experience in a quality, analytical, or operations role reviewing customer conversations, such as CX or QA, support operations, compliance QA, or conversation design • Strong SQL skills • Comfort building and reading dashboards in a BI tool such as Tableau • Ability to apply a written rubric consistently and explain decisions • Strong written communication • Comfort working with sensitive consumer data in a regulated environment • Experience working in Agile/Scrum teams • Experience in consumer finance, credit, banking, fintech, or other regulated environments is a plus • Knowledge of FCRA, ECOA, and UDAAP is a plus • Knowledge of credit reports and FICO Scores is a plus • Experience with conversational AI or LLM-based products is a plus • Experience with data labeling, rubric design, inter-rater agreement, failure-mode taxonomies, basic Python and Jupyter, or Legal and Compliance escalation processes is a plus

🏖️ Benefits

• Highly competitive compensation, benefits and rewards programs • Inclusive culture • Professional development and valuable learning experiences • Work/life balance • Employee resource groups • Social events • Employment and advancement opportunities without regard to protected characteristics

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume