Quality Engineering Consultant – AI/ML Testing

Job not on LinkedIn

🕒 September 2

🇮🇳 India – Remote

⏰ Full Time

🟡 Mid-level

🟠 Senior

🔧 QA Engineer (Quality Assurance)

Where this job comes from

Posted on FindErnest's careers page ↗FindErnest's careers page, not through a job board

First seen by Remote Rocketship September 2

Still open on FindErnest's site. Checked 5 hours ago.

Apply Now
Logo of FindErnest

FindErnest

11 - 50 employees

Founded 2022

💼 Consulting

🏢 Enterprise

🤝 B2B

💰 $20k Pre Seed Round - FindErnest on 2023-01

Consulting • Enterprise • B2B

FindErnest is a global technology consulting and services company that helps enterprises, GCCs, startups, and SMBs accelerate digital transformation through a combination of technology consulting, product/platform engineering, managed IT services, AI & intelligent automation, cloud solutions, cybersecurity, enterprise applications, and workforce/talent solutions. Founded in 2022, the firm emphasizes integrating strategy and execution—combining advisory, implementation, managed services, and staffing under one roof—to deliver measurable business outcomes quickly and cost-effectively. Key offerings include digital transformation, product development, platform engineering, managed IT, AI automation, cloud and data services, and talent solutions; industries served span Technology & Internet, Healthcare & Life Sciences, BFSI, TME, Engineering & Construction, Supply Chain & Manufacturing, and Energy & Utilities. FindErnest positions itself as a mid-market, agile alternative to large SIs and pure staffing firms by delivering end-to-end transformation with a security-first, outcome-driven approach.

📋 Description

• Design and execute test strategies for AI/GenAI applications, including LLM-powered solutions, AI agents, copilots, and conversational systems • Validate AI outputs for accuracy, relevance, groundedness, consistency, hallucination detection, and toxicity and safety compliance • Create and maintain prompt test suites and benchmark datasets • Perform functional, regression, performance, and reliability testing for AI-enabled applications • Define and execute AI evaluation metrics and quality gates • Develop and maintain Playwright test automation frameworks • Automate UI, API, and end-to-end business workflows • Create reusable automation assets and testing accelerators • Integrate automated tests into CI/CD pipelines • Support shift-left testing and quality engineering practices • Design and execute API test cases using Postman, REST Assured, and Playwright API Testing • Validate API contracts, authentication, authorization, error handling, and performance • Conduct integration testing across distributed systems and third-party services • Establish AI evaluation frameworks and quality scorecards • Measure model quality using precision/recall, relevancy scoring, semantic similarity, groundedness validation, and human-in-the-loop evaluation • Analyze AI quality trends and recommend improvements • Support Responsible AI and model governance requirements • Collaborate with engineering, product, and AI teams to identify quality risks early • Participate in requirement reviews and design discussions • Communicate testing progress, risks, and quality metrics to stakeholders • Contribute to QE best practices, standards, and reusable playbooks

🎯 Requirements

• 4 - 6 years of experience in Quality Engineering, Software Testing, or Test Automation • Hands-on experience testing AI/Generative AI applications • Experience with modern web application testing and API testing • Strong expertise in Playwright automation • Hands-on experience with API testing and automation • Experience validating AI/LLM-based applications and AI agents • Good understanding of AI evaluation methodologies and testing techniques • Familiarity with M365 Copilot, OpenAI/Azure OpenAI, Claude, Gemini, and Agentic AI frameworks • Knowledge of test management tools such as Azure DevOps (ADO), Jira, or similar platforms • Experience working with CI/CD pipelines and DevOps practices • Bachelor's degree in Computer Science, Information Technology, Engineering, or a related discipline

🏖️ Benefits

• Remote work arrangement

Apply Now

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Similar Jobs

🕒 August 28

Provenir

201 - 500

☁️ SaaS

💳 Fintech

🤖 Artificial Intelligence

QA Engineer automating web, graph, and AI/ML testing for Provenir’s decision intelligence platform. Building CI/CD quality gates and validating probabilistic systems.

🕒 August 27

Teradyne

5001 - 10000

🏭 Manufacturing

🔧 Hardware

🤝 B2B

Software QA Engineer testing Teradyne web applications and APIs. Designing automated and manual tests while improving quality across the development lifecycle.

🕒 August 12

Med-Metrix

1001 - 5000

🏥 Healthcare

☁️ SaaS

Quality Assurance Analyst testing Med-Metrix software remotely from India during US night hours. Creating test plans, executing manual and automated tests, and managing defects and deployments.

🇮🇳 India – Remote

🔥 Funding within the last year

💰 Private equity on 2025-09

⏰ Full Time

🟡 Mid-level

🟠 Senior

🔧 QA Engineer (Quality Assurance)

🕒 August 10

SupplyHouse.com

501 - 1000

📦 Logistics

🏭 Manufacturing

🛍️ eCommerce

QA Engineer automating web and mobile testing for SupplyHouse.com, an HVAC, plumbing, heating, and electrical e-commerce retailer. Building Cypress and Appium test coverage and validating software quality.

🕒 August 6

Binary Stream Software - ERP Solutions

51 - 200

💼 Consulting

🏥 Healthcare

🏭 Manufacturing

QA Analyst testing cloud-based ERP and financial software for global Microsoft ISV Binary Stream. Executing manual, database, regression, integration, and usability testing while tracking defects.

Built by Lior Neu-ner. I'd love to hear your feedback — Get in touch via DM or support@remoterocketship.com
Follow Remote Rocketship on LinkedInFollow Remote Rocketship on InstagramFollow Remote Rocketship on FacebookFollow Remote Rocketship on TikTok