Senior Data Engineer - Azure Databricks (Telangana)

Job Description

We are looking for a skilled Data Engineer with 5+ years of experience to design, build, and maintain scalable data pipelines and platforms using Azure and Databricks . The ideal candidate will work closely with analytics, data science, and business teams to deliver reliable, high-performance data solutions that enable data-driven decision-making.

Key Responsibilities

• Design, develop, and optimize end-to-end data pipelines using Azure Databricks
• Build and maintain scalable ETL/ELT frameworks for structured and unstructured data
• Ingest data from multiple sources including APIs, databases, files, and streaming systems
• Work with Azure Data Lake Storage (ADLS Gen2) for efficient data storage and retrieval
• Develop data transformation logic using PySpark and Spark SQL
• Ensure data quality, reliability, and performance across pipelines
• Implement data modeling concepts to support analytics and reporting use cases
• Collaborate with Data Scientists, BI teams, and business stakeholders to understand data requirements
• Monitor, troubleshoot, and optimize Databricks jobs for performance and cost efficiency
• Apply best practices around security, governance, and compliance within Azure
• Participate in code reviews, documentation, and continuous improvement initiativesQualifications
• 5+ years of hands-on experience as a Data Engineer
• Strong experience with Azure Databricks
• Proficiency in PySpark , Spark SQL , and Python
• Experience working with Azure Data Lake Storage (ADLS Gen2)
• Solid understanding of data warehousing and data modeling concepts
• Experience building batch and/or streaming data pipelines
• Familiarity with Azure Data Factory or similar orchestration tools
• Strong SQL skills and experience with relational databases
• Experience using Git or similar version control systems
• Solid analytical, problem-solving, and communication skills

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.