Senior Lead Data Engineer - Python/Spark|Gurugram|9+ Yrs

  • HuQuo
  • India, G...
  • 9 - 15 Yrs

Job Description

  • 10+ years of experience in software development and delivery.
  • 5 to 7 years of relevant work experience on large-scale Data applications
  • Good aptitude, strong problem-solving abilities, and analytical skills
  • Should have team handling experience , leading team of 6-7 members OR handle 2-3 smaller teams
  • Should be able to do coding, debugging, performance tuning, and deploying the apps to the Production environment.
  • Should have good working experience on :
  • Hadoop ecosystem (HDFS, Hive, Yarn, File formats like Avro/Parquet)
  • Spark - Batch Processing
  • Setting ETL pipelines
  • Python or Java programming language is mandatory.
  • Worked in data Lake , Data warehousing Domain in Cloud Environment
  • Exposure to Snowflake , Redshift or other Distributed Data Warehousing System
  • Experience working in Agile Methodology
  • Ability to learn and help the team learn new technologies quickly
  • Take up the complete planning/design/implementation/UAT/delivery of the project with strong sense of ownership.
  • Excellent communication and coordination skills
  • Knowledgeable (and preferred hands-on) - UNIX environments, different CICD tools, Scheduling tools like Autosys , Airflow etc.
  • Must be able to integrate quickly into the team and work independently towards team goals.
  • Nice to Have :
  • Experience in any of the cloud (AWS/Azure/GCP) , preferred is AWS
  • Good to have experience in Snowflake

Job Responsibilities

  • Take the complete responsibility for the project execution
  • Be accountable for the delivery of the project in the defined timelines with good quality
  • Define and follow the processes for project execution and delivery.
  • Drive and motivate the team to be agile and follow agile methodology
  • Guide the team technically and give the team a direction on how to plan, design, implement and deliver the projects.
  • Define the architecture and discuss the pros-cons of the same with the team
  • Involve the team during the brainstorming sessions as appropriate.
  • Work with other team leads to get the architecture reviewed.
  • Work with the clients and counterparts (in the US) to drive the project execution.
  • Keep all the stakeholders updated about the project status/risks/issues if there are any.
  • Work closely with the management wherever and whenever required, to ensure smooth execution and delivery of the project.
  • Should be comfortable travel to Pune as and when required
  • Key Skills : Java / Python , Spark, Snowflake, AWS
  • Education : BE/B.Tech from a reputed institute

Location

Gurgaon, Haryana, India