Senior Test Automation Engineer (AI/ML testing)

Job Summary

We are seeking a highly skilled Associate Senior QA Engineer who will function as a senior individual contributor while influencing quality strategy, and best practices across teams. The role demands deep hands-on expertise in UI, API, Database, Automation, and AI/LLM testing, with the ability to design scalable test approaches for modern applications.

Must-Have Skills

• Core QA & Testing Fundamentals:
• Manual testing (functional, regression, smoke, sanity)
• UI/UX validation across browsers & devices
• Strong understanding of test design techniques & defect lifecycle
• Test Strategy & planning: Develop comprehensive test plans and strategies for web applications, APIs, cloud services,
• Test Data Management: Test data creation, management, and cleanup strategies
• Mentoring and leading a team of 2-3 testers
• API Testing
• Hands-on with Postman, REST Assured
• Clear understanding of RESTful APIs, GraphQL, WebSockets
• Ability to validate API contracts, response formats, schema, and error codes
• Automation Testing:
• Experience building or maintaining automated test frameworks in Python/Java/TS/JS
• Automation Tool :Playwright, Selenium
• CI/CD Integration: Integrating automated tests into CI/CD pipelines (Jenkins, GitLab CI, GitHub Actions)
• AI / LLM / Agent Testing:
• Familiarity with LangChain / LangGraph (agent workflows, memory, tools)
• Experience in prompt testing, multi-turn flow validation & LLM response evaluation
• Model evaluation concepts (accuracy, precision, recall, hallucination detection)
• Data validation within ML pipelines

Good-to-Have Skills

• Data Pipeline & Backend Testing
• ETL/ELT validation (Airflow, dbt, Spark)
• Strong SQL for backend/database testing
• Performance /Load Testing
• Load & stress testing (JMeter, k6)
• 2-3 years working on cloud - Azure or AWS
• 3-5 years of Experience working in Healthcare domain

Key Responsibilities

• Define Test Architecture & Strategy :Own the end-to-end test architecture and quality strategy across UI, API, database, and AI-driven systems, ensuring scalability, reliability, and alignment with business goals.
• Lead Automation Framework Design:Architect and govern robust UI and API automation frameworks (Playwright/Selenium), integrated with CI/CD pipelines and designed for long-term maintainability.
• Drive API & Backend Quality:Establish best practices for API, integration, and database testing, including contract validation, data integrity checks, and end-to-end backend workflows.
• Own AI & Agentic System Testing:Define testing approaches for AI agents and LLM-based features, including response validation, hallucination detection, bias checks, prompt behavior, and multi-agent workflows.
• Mentor & Coach QA Teams:Mentor senior testers and automation engineers, review designs and code, and uplift team capability in modern testing, automation, and AI quality practices.