QA Automation Engineer – AI

🔥 16 hours ago

🌲 North Carolina – Remote

🌲 North Carolina – Remote

⏰ Full Time

🟡 Mid-level

🟠 Senior

⚙️ Software Development Engineer in Test (SDET)

🦅 H1B Visa Sponsor

This company has sponsored H1B visas in the past.

👻 Ghost score 11%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Anaplan

WebsiteLinkedInAll Job Openings

1001 - 5000 employees

Founded 2006

💼 Consulting

📦 Logistics

🏥 Healthcare

💰 Secondary Market on 2018-03

Consulting • Logistics • Healthcare

Anaplan is a leading provider of connected planning and business modeling solutions. The platform enables organizations to transform their decision-making processes using intelligent planning, data management, and predictive insights. Anaplan serves a diverse range of industries, including consumer goods, financial services, and manufacturing, by offering tailored solutions that enhance productivity and drive digital transformation. Recognized by Gartner, Forrester, and IDC, Anaplan is a trusted partner for enterprises looking to improve collaboration and gain a competitive edge through agile and comprehensive planning tools.

📋 Description

• Develop and execute end-to-end automated test coverage across evaluations, APIs, UIs, and performance tests using Python • Implement testing strategies for conversational AI, agentic systems, and LLM-powered workflows • Run and maintain automated prompt-testing suites, including regression tests for unintended model-behaviour changes • Support evaluation frameworks measuring GenAI accuracy, relevance, safety, consistency, and latency • Curate high-quality test datasets and golden examples representing diverse user scenarios and edge cases • Assist with adversarial testing for model hallucinations and biases, and support production alerting integration • Use internal tools and testing frameworks to help developers validate GenAI code and assist with user acceptance testing • Own automated coverage for the workstream and document testing procedures and known issues

🎯 Requirements

• Solid QA or test engineering experience, with exposure to or a keen interest in AI/ML systems • Hands-on Python coding experience • Good understanding of GenAI technologies, including LLMs, prompt engineering, and AI application patterns • Solid experience with test automation frameworks and scripting, including Python, Pytest, API, and UI testing • Practical knowledge of functional, integration, regression, performance, and security testing methodologies • Ability to execute test cases and evaluate criteria for non-deterministic systems • Experience with API testing tools such as Postman and REST Assured, plus backend testing • Familiarity with CI/CD pipelines and automated testing integration

🏖️ Benefits

• Inclusive culture focused on diversity, equity, inclusion and belonging • Reasonable accommodation for individuals with disabilities • Equitable benefits and all privileges of employment