
51 - 200 employees
☁️ SaaS
🤖 Artificial Intelligence
🏢 Enterprise
SaaS • Artificial Intelligence • Enterprise
Cresta is an enterprise-grade AI platform that focuses on enhancing contact center operations. By employing a unified platform for human and virtual agents, Cresta aims to improve customer experience, increase revenue, and reduce costs. The platform integrates AI to assist with sales, customer care, retention, and collections, providing real-time guidance and insights. Cresta's AI capabilities include conversation intelligence, agent assistance, quality management, and virtual agents. With a focus on automation and augmentation, Cresta seeks to transform workflows and customer interactions across various industries, including telecommunications, finance, and retail.
🕒 November 12
🌐 Canada, United States – Remote
💵 Estimated: $44,000 - $65,000 / year
⏰ Full Time
🟡 Mid-level
🤖 Artificial Intelligence
👻 Ghost score 23%
Where this job comes from
Posted on Cresta's careers page ↗, not through a job board
First seen by Remote Rocketship November 12
Still open on Cresta's site. Checked 2 hours ago.

51 - 200 employees
☁️ SaaS
🤖 Artificial Intelligence
🏢 Enterprise
SaaS • Artificial Intelligence • Enterprise
Cresta is an enterprise-grade AI platform that focuses on enhancing contact center operations. By employing a unified platform for human and virtual agents, Cresta aims to improve customer experience, increase revenue, and reduce costs. The platform integrates AI to assist with sales, customer care, retention, and collections, providing real-time guidance and insights. Cresta's AI capabilities include conversation intelligence, agent assistance, quality management, and virtual agents. With a focus on automation and augmentation, Cresta seeks to transform workflows and customer interactions across various industries, including telecommunications, finance, and retail.
• Define and maintain evaluation rubrics covering adherence, outcomes, and conversation experience • Write annotation guidelines and failure taxonomies; run calibration sessions and measure inter-rater agreement • Classify and prioritize failure modes by severity and frequency • Review and annotate production conversations weekly, including multilingual evaluation • Trace failures to likely causes and route evidence to the appropriate owners • Build and operate evaluation pipelines combining automated checks, LLM-based judges, and human review • Calibrate automated judges against human labels • Design stratified sampling strategies • Build and maintain golden sets and regression suites • Maintain failure-mode coverage maps and close evaluation gaps • Design evaluations for high-stakes edge cases and upstream failures • Define quality and latency thresholds • Report quality trends and uncertainty to stakeholders and customers • Collaborate with forward-deployed engineers, backend engineers, product managers, and voice/language engineering teams • Document methodology for reproducible evaluation practices
• 3+ years of professional experience in software engineering, applied ML, or data-intensive research roles • At least 1–2 years working hands-on with LLM-based systems in production • Bachelor’s or Master’s degree in Computer Science, Computational Linguistics, Cognitive Science, Statistics, or a related field, or equivalent practical experience • Built and operated an end-to-end evaluation system for a production LLM product • Deep understanding of evaluation methodology, including golden sets, judges, regression suites, and calibration • Understanding of LLM agents, including prompts, tool calls, context handling, and multi-turn state • Strong programming skills in Python and experience with LLM APIs • Ability to write rubrics and measure inter-rater agreement • Ability to design tests from open-ended quality questions • Willingness to listen to calls, label conversations, and build a failure taxonomy • Professional fluency in at least one language other than English • Strong written communication • Completion of a self-paced CoderPad technical assessment may be required after resume review
• Comprehensive medical, dental, and vision coverage with plans to fit you and your family • Flexible PTO to take the time you need, when you need it • Paid parental leave for all new parents welcoming a new child • Retirement savings plan to help you plan for the future • Remote work setup budget to help you create a productive home office • Monthly wellness and communication stipend to keep you connected and balanced • In-office meal program and commuter benefits provided for onsite employees • Equity as part of total compensation
Apply NowImprove your chances of getting an interview by checking your resume score before you apply.
🕒 November 7
Technical Sales Manager driving full-cycle sales of AI-powered products and software services. Translating technical capabilities into business value, ROI, and successful client engagements.
🕒 October 30
AI consultant integrating generative AI, agents, and automation across Invisible Colors’ creative agency workflows. Building practical systems, documentation, and training to improve scalable creative operations.
🕒 October 29
AI Delivery Manager delivering Maestro’s AI underwriting platform to financial institutions. Leading client implementations, underwriter pods, labeling, adjudication, and SaaS delivery for Canada’s mortgage technology company.
🗣️🇫🇷 French Required
🕒 October 29
AI Delivery Manager delivering Maestro, nesto's AI-powered mortgage-underwriting platform. Leading client implementations, underwriter pods, process mapping, labeling, adjudication, and SaaS delivery.
🇨🇦 Canada – Remote
💰 $80M Series C - Nesto on 2022-12
⏰ Full Time
🟡 Mid-level
🟠 Senior
🤖 Artificial Intelligence
🗣️🇫🇷 French Required
🕒 October 22
Applied AI Value Architect driving Celonis enterprise AI transformations across finance and supply chain processes. Orchestrating customer roadmaps, AI-enabled programs, and ROI realization.
🇨🇦 Canada – Remote
💵 $132k - $151k / year
💰 $600M Debt Financing on 2022-08
⏰ Full Time
🟡 Mid-level
🟠 Senior
🤖 Artificial Intelligence