Pyspark+Scala+SQL+Hive+Airflow (Data analyst)

Role Description

Job Specification Senior Data Engineer

Experience Level 5 years

Role Overview We are seeking a strong Senior Data Engineer to join FX Conduct Team This team is responsible for building robust and scalable data assets leveraging cuttingedge technologies including Apache Spark Scala Hadoop Elastic Search and GCP The ideal candidate will bring expertise in big data frameworks handson experience with Hadoop and cloud platforms and a passion for solving complex data challenges in financial domain Candidate must have the ability to understand and develop the logical flow of applications on technical code level Strong interpersonal skills and ability to work in a team and in global environments Candidate should be proactive have learning attitude adjust to work in dynamic work environments

Required Skills And Experience

Big data Expertise 5 years of handson experience with Apache Spark and Scala

Programming Strong Programming skills in Scala and familiarity with other languages like Java is plus

Hadoop Ecosystem Indepth knowledge of Hadoop tools such as HDFS and Hive

Communication Skills Strong verbal and written communication skills in English to convey technical concepts to both technical and nontechnical stakeholders

Unit Testing Strong experience in writing unit tests

Preferred Qualifications

Experience in financial services domain

Knowledge of CICD practices for data engineering

Experience in Elastic search

Experience in Restful API development

Experience in Agile software development methodologies

Exposure to data governance and compliance practices