SDET III – Generative AI QA

August 19

Apply Now

Receive Emails with Similar Jobs

Netomi

WebsiteLinkedInAll Job Openings

Artificial Intelligence • Enterprise • SaaS

Netomi is a company that specializes in providing AI-powered customer experience solutions. Their platform, known as Agentic OS, is designed for enterprise-scale customer service and integrates seamlessly with existing business systems. Netomi's solutions leverage generative AI and large language models to automate over 80% of customer inquiries, enhance customer satisfaction, and reduce support costs. Netomi is trusted by global brands across various industries, offering secure, proactive, and predictive customer care through omnichannel support, including email, chat, messaging, SMS, social media, search, and voice. The company ensures compliance with stringent security standards and data protection regulations.

51 - 200 employees

🤖 Artificial Intelligence

🏢 Enterprise

☁️ SaaS

💰 $30M Series B on 2021-11

📋 Description

• AI-Aware Test Automation - Design and maintain Python/Java-based automation frameworks (Selenium, Playwright, TestNG/JUnit) for web, API, and backend services. • Extend frameworks to test LLM integrations (OpenAI, HuggingFace, RAG pipelines) with prompt validation, hallucination checks, and response consistency tests. • Implement model benchmarking (latency, accuracy, bias/drift detection) for generative AI features. • Quality Infrastructure - Integrate tests into CI/CD pipelines (Jenkins, GitHub Actions) with cloud workflows (AWS/GCP). • Optimize performance testing (JMeter/Locust) for AI endpoints handling high-throughput inference. • Debug flaky tests in (non-deterministic) AI systems. • Leadership & Innovation - Mentor junior engineers on AI testing best practices. • Research tools like LangChain, synthetic data generators, or adversarial testing techniques. • Advocate for ML-specific quality metrics beyond traditional pass/fail.

🎯 Requirements

• 7–9 years in QA automation with strong Python/Java proficiency. • Hands-on experience with Selenium, Playwright, REST Assured, and CI/CD tools (Jenkins, Docker). • Solid understanding of SQL/NoSQL databases and cloud platforms (AWS/GCP). • Exposure to performance testing (JMeter, K6) and scalable test frameworks. • Experience with LLM testing (prompt engineering, output validation, rubric-based grading). • Familiarity with OpenAI APIs, HuggingFace, or LangChain. • Knowledge of synthetic test data generation for edge-case scenarios. • Autonomy – Thrive in fast-paced, AI-driven environments with minimal supervision. • Analytical Mindset – Debug complex failures in probabilistic AI systems. • Communication - Explain technical trade-offs to non-technical stakeholders.