AI Quality Engineer

Job Description

We are looking for Software Quality Assurance Testers with 38 years of experience, with exposure or interest in LLM (Large Language Model) testing and Agents testing. This role involves analyzing business and technical specifications, defining and executing test cases, and capturing and tracking defects.

You will work on Artificial Intelligence initiatives within a multidisciplinary team focused on emerging trends, cutting edge technologies, and their application in financial services for our clients.

This role supports a hybrid work model, allowing you to balance work from home and office based on your needs and role requirements.

What you will be responsible for

As an SQA Tester, you will

Analyze business and technical specifications.

Write functional test cases and perform regression testing.

Report bugs, verify fixes, and maintain the bug tracking system (e.g., Jira).

Maintain and execute automated tests using Selenium with Python.

Create automated test scripts using the Pytest framework.

Automate web services using Postman and/or Python.

Execute test plans for Large Language Models (LLMs), including prompt validation, accuracy checks, and basic safety evaluation.

Support Agents testing for AI-driven workflows to ensure reliability, robustness, and integration quality.

What we value

These skills will help you succeed in this role

Ability to understand customer/business user needs and translate them into test scenarios.

Strong analytical, technical, and problem-solving skills with attention to detail.

Ability to manage time effectively and prioritize tasks while working within deadlines.

Experience with BDD & TDD (Gherkin).

Experience using Jira as an agile management tool.

Experience in using Selenium with Python for test automation.

Experience in testing LLMs, including prompt-response validation and evaluation of model accuracy and relevance.

Exposure to red teaming for identifying vulnerabilities and edge cases in LLMs (preferred but not mandatory at junior level).

Familiarity with evaluation tools such as OpenAI Evals, Ragas, or custom benchmarking suites (good-to-have).

Education & Preferred Qualifications

38 years of software quality assurance testing experience.

Experience with tools like PyCharm, VS Code, Git, Fiddler, Postman, SoapUI, JMeter, Jenkins, Jira.

Experience in CI/CD-based projects.

Hands on experience or strong exposure to LLM testing and Agents testing is highly preferred.

Additional Requirements

Basic understanding of artificial intelligence and machine learning concepts is preferred.

Familiarity with AI governance, bias detection, and safety validation will be an added advantage. Job Description:

We are looking for Software Quality Assurance Testers with 38 years of experience, with exposure or interest in LLM (Large Language Model) testing and Agents testing. This role involves analyzing business and technical specifications, defining and executing test cases, and capturing and tracking defects.

You will work on Artificial Intelligence initiatives within a multidisciplinary team focused on emerging trends, cutting edge technologies, and their application in financial services for our clients.

This role supports a hybrid work model, allowing you to balance work from home and office based on your needs and role requirements.

What you will be responsible for

As an SQA Tester, you will

Analyze business and technical specifications.

Write functional test cases and perform regression testing.

Report bugs, verify fixes, and maintain the bug tracking system (e.g., Jira).

Maintain and execute automated tests using Selenium with Python.

Create automated test scripts using the Pytest framework.

Automate web services using Postman and/or Python.

Execute test plans for Large Language Models (LLMs), including prompt validation, accuracy checks, and basic safety evaluation.

Support Agents testing for AI-driven workflows to ensure reliability, robustness, and integration quality.

What we value

These skills will help you succeed in this role

Ability to understand customer/business user needs and translate them into test scenarios.

Strong analytical, technical, and problem-solving skills with attention to detail.

Ability to manage time effectively and prioritize tasks while working within deadlines.

Experience with BDD & TDD (Gherkin).

Experience using Jira as an agile management tool.

Experience in using Selenium with Python for test automation.

Experience in testing LLMs, including prompt-response validation and evaluation of model accuracy and relevance.

Exposure to red teaming for identifying vulnerabilities and edge cases in LLMs (preferred but not mandatory at junior level).

Familiarity with evaluation tools such as OpenAI Evals, Ragas, or custom benchmarking suites (good-to-have).

Education & Preferred Qualifications