AI QA Engineer – AI Test Architect

Vaga não está no LinkedIn

🕒 6 dias atrás

Candidatar-se

Encontrar Vagas Remotas Similares

Configurar Alerta de Email para Novas Vagas

📊 Verifique sua pontuação de currículo para esta vaga

Melhore suas chances de conseguir uma entrevista verificando sua pontuação de currículo antes de se candidatar.

Enviar Currículo

Togal.AI

SiteLinkedInTodas as Vagas

11 - 50 funcionários

Fundada em 2019

🏗️ Construção

💼 Consultoria

🤖 Inteligência Artificial

💰 $7.000.000 Seed Round - Togal.AI em 2023-03

Construction • Consulting • Artificial Intelligence

Togal. AI é uma plataforma de takeoff baseada em nuvem com tecnologia de IA, voltada para a indústria da construção, que detecta, mede, compara e rotula automaticamente espaços e características em plantas e desenhos arquitetônicos. Desenvolvida por estimadores para estimadores, ela acelera os takeoffs pré-construção, aumenta a precisão, permite a colaboração em tempo real da equipe e automatiza tarefas manuais (incluindo interação em linguagem natural via Togal. CHAT) para ajudar empreiteiros e subcontratados a produzirem propostas mais rápidas e precisas.

Descrição

• Define, implement, and continuously evolve testing standards across functional, non-functional, and AI-specific dimensions from requirements through production • Design and run performance, load, and stress test suites integrated into CI/CD pipelines with release quality gates • Establish and operate LLM/AI evaluation pipelines assessing accuracy, hallucination rate, relevance, faithfulness, and safety • Validate non-deterministic AI outputs, prompt variability, model regression, guardrails, and multi-step agent task completion • Embed QA into planning, design reviews, and sprint ceremonies • Advocate for quality across engineering, product, and AI teams; run blameless post-mortems and define quality metrics • Use AI-assisted tooling, self-healing automation, and intelligent test prioritization • Define and monitor post-release quality signals, model drift indicators, and SLO thresholds

🎯 Requisitos

• Solid understanding of deterministic testing, including test planning, test case design, functional, regression, and exploratory testing, defect lifecycle management, and quality metrics • Deep expertise writing and maintaining automated test suites with Playwright, Cypress, or similar frameworks • Proficiency in at least one modern programming language, especially TypeScript or Python, for building robust test libraries • Hands-on performance, load, and stress testing with k6 or JMeter, including CI/CD integration and threshold-based quality gates • Practical understanding of AI non-determinism and testing LLM-based features for hallucination, consistency, safety, and latency • Understanding of LLM evaluation concepts, including BLEU, ROUGE, and LLM-as-judge patterns • Familiarity with at least one evaluation framework, such as DeepEval or RAGAS • Experience integrating test suites into GitHub Actions, CircleCI, or equivalent CI/CD pipelines • Ability to own quality outcomes, set standards, raise risk flags, and influence cross-functional teams • Understanding of architectural patterns, microservices, and API testing with REST and gRPC using Postman or custom frameworks • Experience with Docker and Kubernetes for scalable testing environments • Nice-to-have: agentic systems testing, AIOps/MLOps/LLMOps, accessibility or security testing, red teaming, adversarial input testing, prompt injection validation, and startup or scale-up experience

🏖️ Benefícios

• Comprehensive benefits • Competitive compensation package • Flexible work arrangements • Culture of innovation, continuous learning, and high growth