Still Struggling with Tech Hiring? Discover Olibr's Solution Now!

Big Data Engineer/Manager - ETL/Python|Pune|5+Years

  • HuQuo
  • India, P...
  • 5 - 7 Yrs

Job Description

  • Tech-savvy engineer willing to learn new skills and track industry trends.
  • 5-7 years of solid data engineering experience, especially in open source, data-intensive, distributed environments, with a minimum of 5 years of experience in big data technologies like Spark, Hive, HBase, and Hadoop.
  • Programming background with mandatory skills in Scala, Spark, and Java/Python.
  • Experience with technologies such as MapReduce, HDFS, YARN, and Spark Streaming.
  • Experience in creating end-to-end pipelines in Hadoop.
  • Proficiency in SQL, Hive, and NoSQL databases.
  • Experience in Linux/Unix platforms.
  • Familiarity with cloud design patterns.
  • Experience with Jenkins, GitHub, debugging code, and error handling.
  • Experience in building end-to-end ETL pipelines.
  • Self-starter with a resourceful personality and ability to manage pressure situations.
  • Experience working with geographically distributed teams.
  • Ability to work on high-level business and technical problems and contribute to architectural decisions.
  • Experience creating core frameworks/templates for team members to start implementation/execution.

Job Responsibilities

  • Build data and ETL pipelines in Hadoop.
  • Support migration of data to the cloud using big data technologies like Spark, Hive, Hadoop, Scala, and Python.
  • Interact with customers daily to ensure smooth engagement.
  • Exposure to Google Cloud Platform (GCP).
  • Responsible for timely and quality deliveries.
  • Exposure to Scrum and Agile development best practices.
  • Share knowledge and experience within other groups in the organization by conducting technical sessions and training.

Location

Pune, Maharashtra, India