Associate Director, AI Product Engineer – Financial Institution Operations & Technology

🔥 0 minutes ago

🇺🇸 United States – Remote

💵 $180k - $250k / year

⏰ Full Time

🟠 Senior

🧑‍💻 Full-stack Engineer

👻 Ghost score 1%

What it means: the chance that no one is really hiring for this job.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Berkeley Research Group (BRG)

WebsiteLinkedInAll Job Openings

1001 - 5000 employees

🏗️ Construction

📦 Logistics

🎖️ Defense

💰 Venture Round on 2020-07

Construction • Logistics • Defense

Berkeley Research Group (BRG) is a global consulting firm that helps leading organizations advance in the fields of corporate finance; economics, disputes, and investigations; and performance improvement. With offices around the world, BRG is an integrated group of experts, industry leaders, academics, data scientists, and professionals working across borders and disciplines. The firm specializes in various sectors, including construction, energy, technology, and healthcare, delivering inspired insights and practical strategies to help clients navigate their challenges effectively.

📋 Description

• Define what each product must get right, how errors could affect customers or operations, and what evidence is needed before use • Build reusable software that runs tests, scores results, and compares versions • Work with engineers to reproduce failures, identify causes, and verify fixes • Contribute product code to improve accuracy and control • Set pass criteria and scoring rules with business and risk teams • Maintain representative test cases, including incomplete records and rare, high-impact errors • Measure factual accuracy, missing information, incorrect additions, and support from cited sources • Check automated scoring against expert review • Compare product versions on the same tests and report results by task and error severity • Verify extracted facts against original records, including amounts, dates, totals, and citation support • Test when the product should answer, request clarification, or refer work to a person • Test for invented facts, unsupported recommendations, unfair outcomes, sensitive-information disclosure, and safeguard circumvention • Protect client data, restrict access and permitted actions, and use only approved services • Document repeatable test results, unresolved failures, and limitations • Recommend release, restricted use, or further development and support independent review and client approvals • Sample live outputs, track error rates and user corrections, and trigger investigation or suspension when quality falls below agreed standards • Explain findings, coach colleagues, develop reusable tests and controls, and support proposals, technical estimates, and recruiting

🎯 Requirements

• Minimum 5 years of working experience, including a minimum of 1 year of experience in AI evaluation • Demonstrated ownership of AI evaluations that informed real release decisions, including test design, expert-reviewed answers, scoring, and failure investigation • Strong Python skills and experience building automated test software, connecting data sources, and recording repeatable results across product versions • Ability to select representative samples, interpret error rates and uncertainty, and distinguish genuine improvement from weak tests or inconsistent scoring • Experience building or improving AI products used in operations, with practical controls for sensitive data, output accuracy, and human review • Experience in financial services or another regulated environment, with ability to translate business requirements into practical tests and controls • Strong communication and teamwork skills, ability to mentor colleagues, and willingness to travel to client sites as needed • Preferred: banking experience in lending, payments, servicing, collections, or default operations • Preferred: experience independently evaluating AI products, checking automated scoring against expert judgment, or testing security and fairness in financial institutions • Preferred: experience evaluating document review, summaries, information search, or automated workflows, including comparisons across AI services and conventional software • Preferred: consulting or product delivery experience, including client discovery, implementation planning, and user adoption • Candidates must be able to verify their legal right to work in the United States without employer sponsorship now or in the future

🏖️ Benefits

• Equal opportunity employment • Reasonable accommodation for essential job functions • Travel opportunities to client sites as needed

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume