Candidate's About
I am a data engineer with 7 years of experience in a cloud environment, building scalable Big Data Analytics Pipeline, and providing technical solutions that meet data warehousing design standards. I have migrated projects from SSIS to Azure Data Factor and created design and mapping documents to support my team members. I have followed Agile Methodologies and efficiently managed code in GitHub.
Work & Experience
BBig Data Engineer
Jun 21 - PRESENT
- Developing pipeline in PySpark and Python
- Deploying code via Jenkins on the server
- Creating a Teradata database for source data and control tables
- Creating Hive tables and views on top of the HDFS location
- Managing code in GitHub
- Following Agile Methodologies
- Supporting production runs and wiki pages documentation related to project
DData Engineer
Mar 19 - Jun 21
- Developed and managed various data pipelines in Postgres and Hive
- Developed AWS S3 bucket for storing data, EC2 instances for running Postgres pipelines, and EMR clusters for executing Hive and PySpark pipeline runs
- Created reports using AWS QuickSight
- Managed code in Bitbucket
- Followed Agile Methodologies
- Maintained confluence pages for projects Read More
EETL Developer
Sep 15 - Mar 19
- Developed pipeline using SSIS
- Migrated project from SSIS to Azure Data Factory
- Created design and mapping documents
- Provided technical and functional support to team members
- Worked closely with the onsite team and fulfilled their requirements on a daily basis
Education
ABachelor of Engineering (BE) in Electronics and Communication
Acharya Institute of Technology