Software Development Engineer in Test – Agentic Automation

🔥 10 minutes ago

🇨🇦 Canada – Remote

💵 $120k - $170k / year

⏰ Full Time

🟡 Mid-level

🟠 Senior

⚙️ Software Development Engineer in Test (SDET)

👻 Ghost score 0%

What it means: the chance that no one is really hiring for this job.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

CrowdStrike

WebsiteLinkedInAll Job Openings

5001 - 10000 employees

Founded 2011

🔒 Cybersecurity

☁️ SaaS

🤖 Artificial Intelligence

Cybersecurity • SaaS • Artificial Intelligence

CrowdStrike is a cybersecurity company that provides cloud-based security services to stop breaches. It is recognized as a leader in endpoint protection, identity and cloud security, and managed detection and response. CrowdStrike's platform, Falcon, integrates artificial intelligence to offer real-time visibility, detection, and protection against sophisticated cyber threats. The company is lauded for its effectiveness in securing networks and data, making it a trusted partner for businesses worldwide.

📋 Description

• Design, develop and maintain automated test suites and frameworks for distributed, cloud-based systems • Maintain and extend agentic test generation and analysis tooling built into major testing frameworks • Build AI-powered workflows and agent skill files to automate manual test toil, including test data setup, failure triage and release validation • Maintain eval suites, golden datasets and scoring pipelines for AI agents, scoring correctness, tool use, trajectory, faithfulness, safety, latency and cost • Enforce CI eval gates so changes to prompts, models, tools or orchestration are measured before release • Feed production failures back into evaluation sets • Detect and mitigate unreliable AI output, including hallucinated assertions, flaky or redundant tests, incorrect failure triage and judge bias • Design, run and analyze performance, load and stress tests to find bottlenecks and validate scalability across distributed services • Validate integration of platform systems, services, data and applications, as well as customer-facing features end-to-end • Review product, system and agent designs for testability, evaluability, flaws and improvements • Share knowledge and contribute to the team's quality and evaluation standards

🎯 Requirements

• Degree in Computer Science or equivalent experience (OOP, data structures, algorithms, distributed systems) • 5+ years in software quality, including 3+ years of hands-on SDET or test automation development • Strong programming skills in at least one of Java, Node.js or Python, with Python preferred for eval and agent work • Proven experience designing, building and maintaining test frameworks integrated with CI/CD pipelines (e.g., Jenkins, GitLab CI/CD) • Experience testing web services and APIs (REST, gRPC, JSON) • Hands-on experience with performance and load testing using tools such as Locust or k6, including designing realistic workloads and interpreting results (latency percentiles, throughput, error rates) • A skeptical, verification-first mindset toward AI output, with a clear sense of when to use deterministic assertions and when to use probabilistic evaluation • Proven experience utilizing AI technologies to enhance decision-making, streamline workflows and processes, improve efficiency and drive business outcomes • Expected to maintain legal entitlement to work in Canada • Subject to background checks, including criminal record, credit, and/or reference checks • Bonus: End-to-end eval-driven development experience, including release gating, trajectory and tool-use evals, and calibrating LLM-as-judge against human review • Bonus: Prompt engineering and context or tool design for agents • Bonus: Integrating performance tests into CI/CD and using observability tools (e.g., Grafana, Prometheus, CloudWatch) • Bonus: Background in cybersecurity, threat intelligence, malware analysis or sandboxing/detonation environments

🏖️ Benefits

• Market leader in compensation and equity awards • Comprehensive physical and mental wellness programs • Competitive vacation and holidays for recharge • Paid parental and adoption leaves • Professional development opportunities for all employees regardless of level or role • Employee Networks, geographic neighborhood groups, and volunteer opportunities to build connections • Vibrant office culture with world class amenities • Variable/incentive compensation • Equity • Equal pay for equal work

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume