QA Engineer – GTM Applications

🔥 0 minutes ago

Apply Now

Find Similar Remote Jobs

Receive Emails For Remote Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

CrowdStrike

WebsiteLinkedInAll Job Openings

5001 - 10000 employees

Founded 2011

🔒 Cybersecurity

☁️ SaaS

🤖 Artificial Intelligence

Cybersecurity • SaaS • Artificial Intelligence

CrowdStrike is a cybersecurity company that provides cloud-based security services to stop breaches. It is recognized as a leader in endpoint protection, identity and cloud security, and managed detection and response. CrowdStrike's platform, Falcon, integrates artificial intelligence to offer real-time visibility, detection, and protection against sophisticated cyber threats. The company is lauded for its effectiveness in securing networks and data, making it a trusted partner for businesses worldwide.

📋 Description

• Define and own the end-to-end test strategy for agentic AI workstreams, establishing quality standards that account for the probabilistic, non-deterministic nature of LLM-powered systems. • Design and implement AI-specific evaluation frameworks covering hallucination detection, prompt quality scoring, agent task completion rates, and output faithfulness against ground-truth references. • Build and maintain automated test suites in Python using frameworks (pytest, robot framework etc.) covering unit, integration, and system-level scenarios across all workstream components. • Develop RAG pipeline test coverage including retrieval precision and recall, semantic relevance scoring, context faithfulness, and end-to-end query-to-answer accuracy using tools such as RAGAS. • Design and Build QA automation tests leveraging industry-standard tools and technologies, encompassing functional, regression, and end-to-end integration testing across connected systems and platforms. • Build and execute Slack integration test suites validating bot response correctness, Workflow Builder trigger fidelity, agentic Slack bot state management, and error handling under edge-case inputs. • Integrate automated tests into CI/CD pipelines (GitHub Actions, Copado, Jenkins) so every pull request is gated by a defined quality bar before merge. • Design and execute performance and load tests for LLM-powered APIs, measuring latency percentiles, token throughput, and degradation patterns under concurrent load. • Conduct security and adversarial testing including prompt injection attempts, output validation for sensitive data leakage, and collaboration with the DevSecOps team on SAST/DAST pipeline findings. • Develop a regression strategy for non-deterministic outputs, defining acceptable variance thresholds, snapshot-based comparisons, and statistical scoring methods that flag genuine regressions without false positives. • Validate observability stack completeness — confirm that distributed tracing, structured logging, SLOs, and AI-specific metrics (latency, token throughput, hallucination rates) are instrumented correctly and alerting as expected. • Collaborate with engineers and the AI Product Owner from requirements grooming through sprint review, contributing testability requirements, acceptance criteria, and definition-of-done checklists. • Own API contract testing across internal and third-party integrations (Salesforce, Marketo, Snowflake, Gong, Clari, G-Suite) using tools such as Postman or REST-assured. • Drive defect lifecycle ownership: triage, severity classification, root cause analysis, regression prevention, and post-release quality retrospectives that feed back into the test strategy. • Champion a shift-left quality culture, coaching engineers to write testable code, instrument their own unit tests, and treat quality as a shared team responsibility rather than a gate at the end of the sprint.

🎯 Requirements

• Bachelor’s degree in Computer Science, Engineering, Information Systems, or a related field • 6+ years of QA or SDET experience, with a track record of building and maintaining automated test frameworks in production environments • Hands-on experience testing AI or ML systems, with a solid understanding of why non-deterministic outputs require different evaluation strategies than conventional software • Strong proficiency in Python for test automation, including designing reusable test utilities, fixtures, mocks, and data factories • Experience with test frameworks and tooling such as pytest, Selenium, Playwright, REST-assured, Postman, or equivalent • Practical understanding of LLM behaviour including temperature effects, token limits, prompt sensitivity, and failure modes that impact test reproducibility • Hands-on Salesforce QA testing experience encompassing validation of Lightning Web Component (LWC) behaviors, Platform Event flows, and API integrations • API testing proficiency across REST endpoints, including contract validation, schema conformance, payload verification, and error-path coverage • Experience integrating automated tests into CI/CD pipelines so quality gates are enforced automatically on every code change • Familiarity with RAG evaluation metrics — faithfulness, answer relevance, context recall — and tooling such as RAGAS or LangSmith for structured AI output evaluation • Experience with performance and load testing tools (Locust, k6, JMeter, or similar) to validate LLM-powered API behaviour under realistic and peak load • Working knowledge of security testing basics: prompt injection, output sanitisation, OWASP top-10 awareness, and coordination with DevSecOps tooling

🏖️ Benefits

• Market leader in compensation and equity awards • Comprehensive physical and mental wellness programs • Competitive vacation and holidays for recharge • Paid parental and adoption leaves • Professional development opportunities for all employees regardless of level or role • Employee Networks, geographic neighborhood groups, and volunteer opportunities to build connections • Vibrant office culture with world class amenities • Great Place to Work Certified™ across the globe