Data Engineering Analyst

Roles and Responsibilities

• Design, develop, test, deploy and maintain large-scale data pipelines using GCP tools such as BigQuery, Cloud Storage, and Pub/Sub.
• Collaborate with cross-functional teams to identify business requirements and design solutions that meet those needs.
• Develop complex data processing workflows using PySpark on Google Cloud Platform (GCP) infrastructure.
• Troubleshoot issues related to data pipeline failures or performance problems.

Job Requirements

• 4-8 years of experience in Data Engineering with expertise in GCP technologies like BigQuery, Cloud Storage, etc.
• Strong proficiency in Python programming language with experience working with libraries like NumPy, Pandas, Matplotlib.
• Experience with big data processing technologies like Hadoop ecosystem (HDFS), Spark (PySpark).
• Knowledge of cloud-based architectures and scalability considerations.