Senior Finance, AI Evaluation Specialist

🔥 0 minutes ago

🗽 New York – Remote

🗽 New York – Remote

💵 $60 - $85 / hour

⏰ Full Time

🟠 Senior

💸 Financial Planning and Analysis (FP&A)

👻 Ghost score 0%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

24-MAG

WebsiteLinkedInAll Job Openings

2 - 10 employees

🤝 B2B

💼 Consulting

B2B • Consulting

24-MAG is a commercial strategy and execution firm that helps B2B organizations design and implement systems, workflows, and operating rhythms for sales, client management, and cross-functional projects. They focus on transforming scattered processes into aligned, measurable, and scalable commercial functions—covering pipeline structure, account management frameworks, and operational discipline for teams seeking efficient, intentional growth.

📋 Description

• Guide research and engineering teams on financial analysis, valuation, investment decisions, capital allocation, and corporate finance • Identify gaps in model understanding across financial statements, transactions, markets, forecasting, and strategic decision-making • Apply practical financial judgment to complex scenarios involving companies, investors, transactions, and commercial objectives • Ensure finance tasks reflect realistic professional decisions and established industry practices • Design challenging, domain-relevant tasks grounded in real financial practice • Write accurate, well-reasoned solutions covering investment banking, asset management, corporate finance, and financial advisory scenarios • Develop problems requiring quantitative analysis, commercial judgment, interpretation of financial information, and evaluation of trade-offs • Ensure tasks are clear, internally consistent, and suitable for structured assessment • Evaluate AI-generated finance responses against established rubrics and scoring criteria • Assess correctness, financial judgment, reasoning quality, relevance, and practical applicability • Compare alternative outputs and determine which response provides the stronger financial analysis • Identify calculation errors, unsupported assumptions, weak financial logic, and incomplete conclusions • Provide clear written feedback to support improvements in model behaviour • Develop and refine evaluation guidelines for financial analysis and decision-making tasks • Define scoring criteria covering technical accuracy, professional judgment, reasoning, and conclusion quality • Participate in calibration activities with other finance specialists • Collaborate across teams to maintain consistency and accuracy throughout the training data • Complete onboarding, specialty calibration, task development, and ongoing quality review

🎯 Requirements

• US-based; candidates must be based in the United States • At least 8 years of dedicated professional experience in finance • Senior-level expertise in investment banking, asset management, corporate finance, financial advisory, or a related area • Experience working within a recognised financial institution, investment firm, professional-services organisation, or comparable company • Prior hands-on experience evaluating LLM or AI-generated outputs using structured rubrics or scoring criteria • Demonstrable career progression into vice president, director, senior manager, or comparable leadership responsibilities • Strong financial judgment and ability to assess complex analytical and commercial decisions • Excellent written and verbal communication skills • Reliable availability for at least 35 hours per week during weekdays • Prior experience evaluating AI or LLM outputs against structured rubrics is required • Applicants should clearly describe relevant AI evaluation experience in their application • A degree in finance, economics, accounting, business administration, mathematics, or a related field is highly relevant • Graduate-level education in finance, economics, business, or management may be helpful • Equivalent senior professional experience in financial analysis, transactions, or investment management may also be considered • Professional credentials such as CFA, CPA, CA, or comparable qualifications may be valuable • Experience with valuation, financial modelling, mergers and acquisitions, capital markets, or portfolio analysis • Background in transaction advisory, private equity, equity research, strategic finance, or investment management • Experience creating structured scoring rubrics, evaluation guidelines, or quality frameworks • Familiarity with model training, human-feedback workflows, annotation, or AI quality assurance • Strong understanding of financial statements, forecasting, risk, capital structure, and investment analysis • Experience reviewing financial models, investment recommendations, transaction materials, or strategic finance proposals • Previous collaboration with product, analytics, research, engineering, or data-science teams • Immediate availability is preferred

🏖️ Benefits

• Full-time W-2 contingent employment arrangement • Fully remote work • Competitive hourly compensation • At least 35 hours per week during weekdays • Immediate availability preferred • Flexible project scope and duration according to programme requirements and performance