Data Engineering-4 plus Years-Noida

Azure Data Engineer | Azure Databricks | ADF | Python | PySpark | SQL

Summary

We are looking for an experienced Azure Data Engineer with strong expertise in Azure Databricks, Azure Data Factory (ADF), Python, PySpark, and SQL. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines while ensuring high data quality, reliability, and performance. This is an excellent opportunity to work on enterprise-scale cloud data platforms and contribute to data-driven business solutions.

Location-Noida

Your Future Employer

Our client is a leading organization committed to driving digital transformation through cutting-edge cloud technologies and advanced data engineering solutions. You will work in a collaborative environment focused on innovation, continuous learning, and delivering scalable enterprise data platforms.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using Azure Data Factory (ADF) and Azure Databricks.
• Develop efficient data transformation workflows using PySpark and Python.
• Build, optimize, and maintain batch and incremental data processing pipelines.
• Write high-performance SQL queries, stored procedures, and views for data extraction, transformation, and reporting.
• Develop reusable Python scripts for automation and data processing.
• Integrate data from multiple sources, including databases, APIs, cloud storage, and flat files.
• Optimize Spark jobs for performance, scalability, and cost efficiency.
• Implement data quality checks, logging, monitoring, exception handling, and alert mechanisms.
• Work with Delta Lake, partitioning strategies, and data optimization techniques.
• Collaborate with business stakeholders and cross-functional teams to understand data requirements and deliver reliable data solutions.
• Troubleshoot production issues, perform root cause analysis, and implement preventive measures.
• Follow coding standards, version control practices, and CI/CD methodologies to ensure high-quality deliverables.

Requirements

• Strong hands-on experience with Azure Databricks.
• Proven expertise in Azure Data Factory (ADF).
• Strong programming experience in Python.
• Excellent coding skills in PySpark for large-scale data processing.
• Advanced SQL knowledge with query optimization and performance tuning.
• Experience with Delta Lake and Spark SQL.
• Hands-on experience with Azure Data Lake Storage (ADLS Gen2).
• Experience using Git or Azure DevOps for source code management.
• Good understanding of data warehousing concepts and dimensional modeling.
• Experience integrating data using REST APIs.
• Exposure to Azure Synapse Analytics will be an added advantage.
• Knowledge of Microsoft Fabric is preferred.
• Experience with Spark Structured Streaming or Apache Kafka is desirable.
• Exposure to Power BI is preferred.
• Familiarity with Azure DevOps CI/CD pipelines.
• Understanding of Agile/Scrum methodologies.

What is in it for you

• Opportunity to work on enterprise-scale Azure cloud data engineering projects.
• Exposure to modern cloud technologies including Azure Databricks, ADF, Delta Lake, and Azure Synapse Analytics.
• Collaborative and technology-driven work environment.
• Continuous learning and career growth opportunities.
• Competitive compensation and comprehensive employee benefits.

Reach us

If you think this role is aligned with your career, kindly write to us along with your updated CV at sadaf.zenab@crescendogroup.in for a confidential discussion on the role.

Disclaimer

We specialize in Senior to C-level niche recruitment. We are passionate about empowering job seekers and employers with an engaging and memorable job search and leadership hiring experience. We do not discriminate on the basis of race, religion, color, origin, gender, sexual orientation, age, marital status, veteran status, or disability status.

Note

We receive a large number of applications every day, making it difficult to respond to each applicant individually. If you do not hear from us within one week, please consider that your profile has not been shortlisted for this opportunity. We sincerely appreciate your patience.

Fraudsters may misuse our name to circulate fake job offers. We never ask candidates for money, purchases, or system upgrades during the recruitment process. Please verify all opportunities through official channels and report any suspicious activity immediately. Stay alert.