AI Solutions Architect

Job not on LinkedIn

🔥 0 minutes ago

🇺🇸 United States – Remote

⏰ Full Time

🟡 Mid-level

🟠 Senior

💻 Solutions Engineer

👻 Ghost score 11%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Testlio

WebsiteLinkedInAll Job Openings

201 - 500 employees

Founded 2013

💼 Consulting

🏥 Healthcare

📦 Logistics

💰 $12M Series B on 2021-10

Consulting • Healthcare • Logistics

Testlio is a leading software testing company that provides comprehensive solutions to enhance the quality and reliability of digital products. Specializing in both manual and automated testing, Testlio serves a diverse range of industries including finance, health, media, and travel. With a cutting-edge platform that integrates seamlessly with DevOps tools, Testlio harnesses data-driven insights to optimize testing processes and deliver superior results. Their extensive global network of testers ensures robust coverage across multiple devices, languages, and payment methods, crucial for businesses aiming to scale globally. Focused on flexibility, speed, and customer satisfaction, Testlio is the trusted partner for ensuring software excellence and reliability.

📋 Description

• Define and maintain the AI service catalog, including offerings, tiers, packaging, and delivery methodology • Track, analyze, and present key metrics demonstrating quality, efficiency, and ROI to stakeholder leadership • Develop and evangelize Testlio's viewpoint on the future of AI Testing • Produce white papers, articles, posts, speeches, and webinars • Collaborate with sales and engagement teams on pre-deal discovery, scoping, and communicating Testlio's AI testing value proposition • Use market and competitive intelligence to refine offerings and provide Product Marketing with positioning details • Deconstruct client AI stack components, including autonomous agents, system prompts, foundation models, and RAG architectures • Formulate tailored AI testing approaches, inform test strategy, design evaluations, and provide technical advisory • Serve as an escalation point and subject-matter expert for Delivery during complex or new client engagements • Solve technical roadblocks, stand up environments, and develop scalable, repeatable AI testing and evaluation solutions • Advance Testlio's proprietary AI testing framework using deterministic code graders, LLM-as-a-judge, and human-in-the-loop validation • Develop templates, curated training datasets, and comprehensive documentation to enable high-quality test execution • Collaborate with data scientists, machine learning engineers, executive sponsors, sales, delivery, and internal teams

🎯 Requirements

• Extensive practical experience developing, training, or fine-tuning machine learning models, NLP structures, or data pipelines • Deep familiarity and hands-on experience with modern LLM evaluation frameworks, such as DeepEval and RAGAs • Direct experience working with, deploying, or testing autonomous agent architectures, planning patterns, or multi-agent swarms, such as LangGraph, AutoGen, and CrewAI • Strong software engineering foundations, including proficiency with complex non-deterministic and deterministic systems • Solid track record leading discovery calls, managing stakeholder workshops, and consulting on real-world enterprise architectures • Experience designing service offerings, understanding pricing dynamics, and formulating go-to-market strategies within professional services or SaaS • Exceptional ability to connect technical data science outputs with clear, strategic customer business value and communicate with engineers and executives • High adaptability to fast-evolving client environments, unvetted sources, and non-deterministic AI application lifecycles • Passion for continuous learning, documentation, and sharing architectural best practices • Ability to work effectively in a fully remote environment

🏖️ Benefits

• Flexible paid time off, including national holidays, personal days, and sick days • Stock options • $300 annual learning stipend for personal and professional development • Fully remote work in a globally distributed team • Tools and guidance to succeed in a fully remote setting • Inclusive, purpose-driven culture • Global enterprise subscription-based business with exceptional clients