Senior Software Developer in Test, Python

🔥 0 minutes ago

Apply Now

Find Similar Remote Jobs

Receive Emails For Remote Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

SOFTSWISS

WebsiteLinkedInAll Job Openings

1001 - 5000 employees

🎮 Gaming

Gaming • Software Development

SOFTSWISS is a leading provider of iGaming software solutions, offering a wide range of products including iGaming Platform, Sportsbook Platform, Game Aggregator, Jackpot Aggregator, and more. Established in 2009, SOFTSWISS has grown to employ over 1,700 people across 4 international offices in Poland, Georgia, Malta, and several remote locations. The company is known for its innovation in the iGaming industry, being the first Crypto iGaming provider in the world. SOFTSWISS not only promotes a dynamic and flexible work culture with comprehensive employee benefits but also commits to social responsibility by supporting various campaigns and initiatives.

📋 Description

• Analyze requirements and define the testing approach for new features and product changes • Automate test cases using the existing framework based on Python and PyTest • Build automated quality evaluation pipelines for AI systems using metrics and LLM-as-judge approaches • Test MCP servers, tool schemas and tool-call behavior, including edge cases and invalid arguments • Evaluate agentic workflows, including tool selection, multi-step reasoning, error handling, loop recovery and state correctness • Maintain and improve the test automation framework and contribute to internal testing tools, including mocks • Prepare and maintain test documentation, including checklists, test cases and quality reports • Participate in test design, estimations, release testing and product quality assessment • Contribute to CI/CD and QA process improvements • Design and maintain evaluation suites and golden datasets for RAG and agentic flows • Perform adversarial testing for AI systems, including prompt injection, jailbreaks, tool misuse and data leakage risks • Set up regression checks for changes in prompts, models, retrieval settings and chunking strategies • Track AI system quality together with cost, latency and token usage • Use tracing and observability tools to debug, measure and improve LLM application behavior

🎯 Requirements

• 5+ years of experience in Quality Assurance, including both manual and automation testing • Solid understanding of QA principles, test design, test coverage, test pyramid and SDLC • Experience with Python-based test automation frameworks, such as PyTest, Behave or similar • Experience with CI/CD and monitoring or alerting tools, such as Datadog, ELK, Sentry or similar • Interest in testing AI/LLM-based systems. Hands-on experience is preferred, but we are also open to candidates who can learn quickly and want to grow in this area • Familiarity with RAG, LLM evaluation and quality metrics, such as groundedness, faithfulness, answer relevance and retrieval quality • Experience or interest in AI evaluation tools, such as RAGAS, DeepEval, promptfoo, LangSmith Eval, TruLens, Arize Phoenix or similar • Understanding of how to test non-deterministic systems, where there may be no single correct output • Familiarity with LangChain, LangGraph, MCP, vector databases, semantic search or LLM observability tools would be a strong plus • Good spoken and written English (B2 level or higher)

🏖️ Benefits

• Full-time work opportunities • Private insurance • An additional Day Off (1) per calendar year • Sports program compensation • Comprehensive Mental Health Programme • Free online English lessons with native speakers • Generous referral program • Training, internal workshops, and participation in international professional conferences and corporate events