AI QA Engineer – AI Test Architect

Job not on LinkedIn

🕒 August 4

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Togal.AI

WebsiteLinkedInAll Job Openings

11 - 50 employees

Founded 2019

🏗️ Construction

💼 Consulting

🤖 Artificial Intelligence

💰 $7M Seed Round - Togal.AI on 2023-03

Construction • Consulting • Artificial Intelligence

Togal. AI is a cloud-based, AI-powered takeoff platform for the construction industry that automatically detects, measures, compares and labels spaces and features on architectural plans and drawings. Built by estimators for estimators, it speeds pre-construction takeoffs, increases accuracy, enables real-time team collaboration, and automates manual tasks (including natural-language interaction via Togal. CHAT) to help contractors and subcontractors produce faster, more accurate bids.

📋 Description

• Define, implement, and continuously evolve testing standards across functional, non-functional, and AI-specific dimensions from requirements through production • Design and run performance, load, and stress test suites integrated into CI/CD pipelines with release quality gates • Establish and operate LLM/AI evaluation pipelines assessing accuracy, hallucination rate, relevance, faithfulness, and safety • Validate non-deterministic AI outputs, prompt variability, model regression, guardrails, and multi-step agent task completion • Embed QA into planning, design reviews, and sprint ceremonies • Advocate for quality across engineering, product, and AI teams; run blameless post-mortems and define quality metrics • Use AI-assisted tooling, self-healing automation, and intelligent test prioritization • Define and monitor post-release quality signals, model drift indicators, and SLO thresholds

🎯 Requirements

• Solid understanding of deterministic testing, including test planning, test case design, functional, regression, and exploratory testing, defect lifecycle management, and quality metrics • Deep expertise writing and maintaining automated test suites with Playwright, Cypress, or similar frameworks • Proficiency in at least one modern programming language, especially TypeScript or Python, for building robust test libraries • Hands-on performance, load, and stress testing with k6 or JMeter, including CI/CD integration and threshold-based quality gates • Practical understanding of AI non-determinism and testing LLM-based features for hallucination, consistency, safety, and latency • Understanding of LLM evaluation concepts, including BLEU, ROUGE, and LLM-as-judge patterns • Familiarity with at least one evaluation framework, such as DeepEval or RAGAS • Experience integrating test suites into GitHub Actions, CircleCI, or equivalent CI/CD pipelines • Ability to own quality outcomes, set standards, raise risk flags, and influence cross-functional teams • Understanding of architectural patterns, microservices, and API testing with REST and gRPC using Postman or custom frameworks • Experience with Docker and Kubernetes for scalable testing environments • Nice-to-have: agentic systems testing, AIOps/MLOps/LLMOps, accessibility or security testing, red teaming, adversarial input testing, prompt injection validation, and startup or scale-up experience

🏖️ Benefits

• Comprehensive benefits • Competitive compensation package • Flexible work arrangements • Culture of innovation, continuous learning, and high growth