Pyspark Developer

Job Summary

We are seeking an experienced Pyspark Developer to join our team. As a Pyspark Developer, you will be responsible for designing, developing, and implementing data processing and analytics solutions using Pyspark, Big Data, and AWS technologies.

Responsibilities

Develop and maintain data processing and analytics solutions using Pyspark, Spark, and Big Data technologies

Design and implement data pipelines using AWS EMR, S3, IAM, Lambda, SNS, and SQS

Collaborate with cross-functional teams to understand business requirements and deliver customized solutions

Optimize and troubleshoot data processing and analytics solutions

Develop and maintain technical documentation for data solutions

Requirements

Good work experience on Big Data Platforms like Hadoop, Spark, Scala, Hive, Impala, SQL

Experience working on Data Engineering projects

Good understanding of SQL

Good understanding of Unix and HDFS commands

Experience working on Data Analytics and Pyspark