Home/Job List/QA Engineer AI Analytics / Databricks Genie

QA Engineer AI Analytics / Databricks Genie

PRI INDIA IT SERVICES PRIVATE LIMITED

Bengaluru, Karnataka, India
Full-Time
Posted 7 days ago

Job Description & Responsibilities

Role Overview

We are looking for a QA Engineer to validate AI-powered analytical solutions where Agents use Databricks Genie to interpret natural-language questions and generate data-driven insights. The role will focus on validating data accuracy, response quality, business logic, and end-to-end reliability.

Key Responsibilities

  • Test AI Agents and analytical solutions powered by Databricks Genie.
  • Validate natural-language questions and Genie-generated analytical responses against source data and SQL results.
  • Test analytical scenarios including KPIs, aggregations, trends, comparisons, filters, and drill-downs.
  • Validate responses across different question phrasings, ambiguous queries, edge cases, and multi-turn conversations.
  • Identify incorrect calculations, inconsistent responses, hallucinations, and incorrect interpretation of business terminology.
  • Perform functional, regression, API, integration, and data validation testing.
  • Build reusable test cases and automated regression suites for analytical queries.
  • Collaborate with Data Engineers, AI Engineers, business SMEs, and Product Owners on acceptance criteria and defect resolution.

Required Skills

  • 3-6 years of QA/testing experience, preferably in data, analytics, BI, or AI solutions.
  • Strong SQL and data validation skills.
  • Experience testing analytical or data-driven applications.
  • Experience with API testing using tools such as Postman.
  • Automation experience using Python/PyTest or similar frameworks.
  • Good understanding of data pipelines, KPIs, aggregations, and analytical datasets.
  • Exposure to Databricks, Databricks Genie, LLMs, or GenAI testing is highly preferred.
  • Understanding of AI testing concepts such as accuracy, groundedness, hallucination, relevance, and response consistency.

Good to Have

  • Azure Databricks / PySpark knowledge.
  • Experience testing conversational BI or natural-language-to-SQL solutions.
  • Experience automating comparison of Agent responses against expected SQL/data outputs.

Required Skills

PythonAzureSQLNLP

Job Details

Employment TypeFull-Time
Work ModeOn-Site
Experience3 – 6 years
Positions1

Posted by

N/A

Posted on:

11 Nov 2026

About PRI INDIA IT SERVICES PRIVATE LIMITED

More open roles

Browse all jobs →