Big Data Engineer Bengaluru (India)

Experience: 4 - 7 Years Location: Bangalore Mandatory Skills: Python, Scala, Spark, Hadoop, Hive, HDFS, Mapreduce, Yarn and Hadoop Architect. Job Summary: We are seeking a highly skilled Big Data Engineer with solid expertise in Spark & Scala, along with solid experience in Java 8, Python (PySpark), and Hadoop architecture. The ideal candidate should possess advanced problem-solving abilities and a strong foundation in building scalable data processing systems. Key Responsibilities ● Design, develop, and optimise large-scale data processing pipelines using Spark and Scala. ● Work with distributed systems and components of the Hadoop ecosystem to handle big data workloads. ● Write efficient, reusable, and scalable code in Java 8 and Python (PySpark). ● Apply advanced logic-building skills to solve complex data engineering problems. ● Collaborate with cross-functional teams, including data scientists, analysts, and product teams. ● Ensure data quality, performance tuning, and reliability of data pipelines. ● Participate in code reviews, testing, and deployment processes. Required Skills ● Robust hands-on experience with Apache Spark (Scala & PySpark) ● Proficiency in Scala programming ● Experience with Java 8 ( positive to have ) ● Strong advanced logic-building and problem-solving skills ● Expertise in Python (PySpark) ● In-depth understanding of Hadoop Architecture and ecosystem (HDFS, YARN, MapReduce