Associate Technical Lead | Data Engineer

Role & responsibilities

• Review and understand business requirements ensuring that development tasks are completed within the timeline provided and that issues are fully tested with minimal defects
• Partner with a software development team to implement best practices and optimize the performance of Data applications to ensure that client needs are met at all times
• Collaborate across the company and interact with our customers to understand, translate, define, design their business challenges and concerns into innovative solutions
• Research on new Big Data technologies, assessing maturity and alignment of technology to business and technology strategy
• Work in a rapid and agile development process to enable increased speed to market while maintaining appropriate controls

Preferred candidate profile

• BE/B.Tech/MCA with At least 6+ years of experience in design and development using Data Engineering technology stack and programming languages
• Mandatory experience in following areas:
• Python/Scala/Java
• Hadoop, HDFS, MR
• Spark SQL, Dataframes, RDDs
• SQL
• Hive / Snowflake/SQL Server/Databricks
• Elastic Search
• Preferred experience in 3 or more of the following areas:
• Spark Streaming, Spark ML
• Kafka/Flume
• Apache NiFi
• Apache Airflow/Oozie
• Cloud-based Data Platforms
• NoSQL Databases HBase/Cassandra/Neo4j/MongoDB
• Good knowledge of the current technology landscape and ability to visualize industry trends
• Working knowledge of Big Data Integration with Third-party or in-house built Metadata Management, Data Quality, and Master Data Management solutions
• Active community involvement through articles,blogs or speaking engagements at conferences