Data Engineering Lead

Join Verdantas - A Top 81 Firm, We at Verdantas, seeking a Data Engineer Lead in Pune to architect and implement scalable, secure AI/ML data solutions aligned with business objectives. The role involves designing robust batch and real-time data architectures, leading ETL/ELT pipeline development, managing data lakes and warehouses, ensuring data quality and governance, mentoring data engineers, collaborating with stakeholders, and deploying cloud-based data infrastructure with CI/CD and cost optimization. Candidates should have 8+ years in data engineering with leadership experience, strong skills in SQL, Python, distributed processing frameworks, and cloud-native data services, with preferred certifications including Google Professional Data Engineer, Azure Data Engineer Associate, and AWS Certified Data Analytics Specialty.

The position requires in-office work in Pune, Maharashtra, India.

Key Responsibilities Architecture & Strategy - Design and implement robust data architectures (batch and real-time) using modern data platforms.

• Define data engineering standards, best practices, and governance policies.
• Evaluate and recommend tools and technologies for data ingestion, storage, processing, and orchestration. Development & Operations - Lead the development of ETL/ELT pipelines using tools like Apache Spark, Airflow, dbt, or Azure Data Factory.
• Build and manage data lakes, data warehouses, and data marts.
• Ensure data quality, lineage, and observability across pipelines. Team Leadership - Mentor and guide data engineers, fostering a culture of technical excellence and continuous improvement.
• Collaborate with data scientists, analysts, and business stakeholders to understand data needs and deliver solutions.
• Manage project timelines, resource allocation, and delivery milestones. Cloud & DevOps Integration - Deploy and manage data infrastructure on cloud platforms (AWS, Azure, GCP).
• Implement CI/CD pipelines for data workflows and infrastructure as code (IaC).
• Optimize performance and cost of data systems.

Skills & Qualifications - Bachelor's or master's in computer science, Data Engineering, or related field.

• 8+ years of experience in data engineering, with 2+ years in a leadership role.
• Strong expertise in SQL, Python, and distributed data processing frameworks (Spark, Flink).
• Experience with cloud-native data services (e.g., AWS Glue, Azure Synapse, BigQuery).
• Familiarity with data modeling, data governance, and metadata management.
• Excellent problem-solving, communication, and stakeholder management skills.

Preferred Certifications

• Google Professional Data Engineer
• Microsoft Certified: Azure Data Engineer Associate
• AWS Certified Data Analytics - Specialty Location and Work Set-up Pune, Maharashtra, India Work Mode: In Office