Data Engineer (GCP)

Responsibilities

• Design and develop scalable data pipelines on GCP.
• Build ETL/ELT solutions using Dataflow and Dataproc.
• Develop and optimise BigQuery datasets for reporting and analytics.
• Manage structured and unstructured data in GCS.
• Implement data quality, validation, and monitoring frameworks.
• Collaborate with business, analytics, and engineering teams to deliver data solutions.
• Troubleshoot production issues and optimise pipeline performance.

Requirements

• Strong SQL expertise.
• Python and PySpark development.
• Apache Spark ecosystem.
• BigQuery, Dataflow, Dataproc, GCS (mandatory).
• Data warehousing and data modelling concepts.
• Experience with batch and real-time data processing.
• Understanding of cloud security and IAM.