QA Engineer

Job not on LinkedIn

🕒 May 27

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Titan AI

WebsiteLinkedInAll Job Openings

1 - 10 employees

Building mobile RPG Games with AI ✨ Companions 🤖. First game: Hell Rush is coming in July 2024.

📋 Description

• Titan builds AI software for banks: purpose-built small language models, a banking ontology, and AI bankers that financial institutions can trust. • You are coming in to do the work: write the test cases, build the evaluation framework, set up CI/CD gates, and triage bugs alongside engineering. • You personally design and execute the evaluation framework for LLM and agentic AI outputs across Foundry, Agent Builder, and client-deployed instances. • You write the assertions, define the behavioral contracts, and own regression baselines for model behavior. • You write and maintain the automated test suite: end-to-end, integration, and regression coverage for backend APIs, document ingestion pipelines, AI inference workflows, and frontend surfaces. • You produce the test artifacts, audit logs, and process documentation that meet SOC 2 Type II standards.

🎯 Requirements

• Seven or more years in software QA engineering, with at least two years personally testing AI or ML systems. • You have written test cases against LLM outputs, built evaluation pipelines from scratch, and know the difference between a flaky test and a genuinely non-deterministic system. • You are fluent in Python and have built automated suites using pytest, Playwright, or Selenium. • You have hands-on experience with RAGAS, DeepEval, LangSmith, or comparable evaluation tooling—not just familiarity with the names. • You can trace a failure from the application layer to infrastructure and know enough about Azure, async systems, and REST APIs to do it without waiting on an engineer to walk you through it. • You have integrated QA gates into CI/CD pipelines and owned the process end to end. • Experience in fintech, banking, or another regulated environment is a strong advantage. • Familiarity with document processing pipelines, multi-agent architectures, RAG validation, or observability tooling such as Arize or Langfuse puts you ahead.

🏖️ Benefits

• Competitive base and meaningful equity. • Remote (US). Occasional travel to client sites and team offsites.