Senior Data Engineer – Azure Databricks

About the Company

Greetings from Teamware Solutions a division of Quantum Leap Consulting Pvt. Ltd

Teamware Solutions (a division of Quantum Leap Consulting Pvt. Ltd.) is a 20+ year industry leader providing IT staffing and consulting services, supporting 4000+ professionals across India, the USA, the Middle East, and APAC for leading GCC clients.

About the Role

We are hiring a Senior Data Engineer – Azure Databricks

• Location: Bangalore
• Work Model: Hybrid
• Experience: 5+ yrs of experience
• Notice Period: Immediate - 15 days
• Shift timings: 12 pm-10 pm

Senior Data Engineer

Exp - 5-8 years exp

• Azure Databricks: Strong hands-on experience with Databricks Lakehouse Architecture, Delta Lake, Unity Catalog, Databricks Apps, Workflows/Jobs, and Databricks Asset Bundles (DAB).
• Real-Time Data Ingestion: Experience with Azure Event Hubs, Spark Structured Streaming, Auto Loader, Databricks Zerobus Ingest, and Delta Change Data Feed (CDF).
• Python & SQL: Strong proficiency in Python, PySpark, and SQL for data engineering, transformation, optimization, and automation.
• Configuration-Driven Framework: Experience designing reusable, modular, metadata-driven Python frameworks with configurable business rules, execution engines, and orchestration layers.
• Lakebase (PostgreSQL): Hands-on experience with Lakebase for operational data storage, relational data modeling, SQL development, and Lakebase-to-Delta synchronization.
• Databricks Lakehouse: Experience implementing Bronze, Silver, and Gold architectures, Delta tables, data transformations, and scalable analytical pipelines.
• Data Pipeline Engineering: Ability to design batch and streaming pipelines with error handling, retries, replay, reconciliation, and failure recovery mechanisms.
• Data Quality & Governance: Experience implementing validation frameworks, data quality checks, quarantine handling, audit logging, and Unity Catalog governance.
• Data Modeling & Historical Processing: Strong understanding of relational data modeling, source-to-target mappings, CDC, SCD Type 2, and historical data synchronization.
• Performance Optimization: Ability to optimize Spark workloads, SQL queries, Delta tables, job execution, and data processing costs.
• Architecture & Collaboration: Ability to translate functional requirements into scalable technical designs, develop data models and DDL scripts, and collaborate with architects, developers, DevOps, and QA teams

Please let me know if you are interested in this position and send me your resume to netra.s@twsol.com