Senior Data QA Engineer – ETL Testing, Data Validation & Automation

Role: Senior Quality Engineer - ETL Testing, Data Quality Testing, and Test Automation

7+ years

Summary

We are seeking a highly technical QE Tester to lead quality assurance for our data-driven ecosystem

• Core Responsibilities
• End-to-End Data Validation: Lead testing for complex data pipelines, ensuring integrity from source systems through the Medallion Architecture (Bronze, Silver, Gold layers).
• Advanced SQL Testing: Write sophisticated queries to validate complex transformations, including advanced joins, window functions, and SCD (SCD1 vs SCD2) logic.
• LLM & Agentic Evaluation: Develop and execute frameworks to detect LLM hallucinations, bias, and non-deterministic output inconsistencies.
• Performance & Governance: Conduct stress testing on AI agents, monitor token costs, and validate guardrails to ensure safe and cost-effective AI operations.
• Collaborative Quality: Partner with Data Engineers and AI Researchers to define "Golden Datasets" and establish business-rule validation criteria.
• Technical Requirements

Data Engineering Validation

• Advanced SQL: Expertise in complex analytical queries, window functions, and validating Slowly Changing Dimensions (SCD).
• Warehousing Fundamentals: Deep understanding of Fact vs Dimension validation and Source-to-Target (STT) reconciliation.
• Pipeline Integrity: Proven experience with Incremental vs Full Load testing, row count reconciliation, and data completeness checks.
• Data Quality (DQ): Ability to design automated DQ checks, data profiling, and validation of business-specific transformation rules.
• API Testing: Proficient in validating RESTful APIs and alternative integration patterns.

LLM & Generative AI Validation

• Prompt Engineering: Validating system prompts, user prompts, and few-shot learning impacts on model behavior.
• Hallucination & Bias Detection: Implementing techniques to identify factual inaccuracies and social biases in model responses.
• Model Parameters: Testing the impact of Temperature, Top-P, and Context Window limits on output quality.
• Agentic Testing: Validating multi-step reasoning, human-in-the-loop workflows, and stress testing autonomous AI agents.
• Benchmarking: Evaluating model performance against Golden Datasets using metrics like BLEU, ROUGE, or semantic similarity.
• Preferred Qualifications
• Experience: 6+ years in Data Quality, ETL Testing, or AI Assurance.
• Tools: Familiarity with dbt, Great Expectations, or LLM evaluation frameworks (LangChain,N8N,).
• Cloud Environments: Hands-on experience with modern data stacks on Azure.
• Mindset: A "non-deterministic" testing approach—comfortable working with distributions and probabilistic outputs rather than just True/False outcomes.