Still Struggling with Tech Hiring? Discover Olibr's Solution Now!
Login / RegisterDeveloperClient
Login/Register

AASHISH PATIL

Data Engineer

  • Pune, MH
  • 3 years

Available From: Available

    Candidate's About

    Highly qualified Data Engineer with 3+ years of experience in the IT industry. Experienced in Development, PySpark, and AWS. Skilled in data extraction, building optimized big data pipelines, and processing structured and unstructured data. Proficient in Spark, SQL, schedulers like Oozie, and file compression techniques. Strong problem-solving abilities and a collaborative mindset.

    Work & Experience

    D

    Data Engineer

    Nov 19 - PRESENT

    Technology

    • Icon
      Hadoop
    • Icon
      Kafka
    • Icon
      AWS
    • Icon
      Python
    • Icon
      Linux
    • Icon
      Power BI

    Tools

    • Icon
      Jira
    • Icon
      Github
    • Designing and developing scalable data pipelines using Hadoop, Spark, and Kafka.
    • Building and maintaining real-time and batch processing systems for data transformation and analysis.
    • Creating and managing data models, data marts, and data warehouses for business intelligence and analytics.
    • Collaborating with cross-functional teams to identify data needs and prioriti...
    Read More

    Retail store Data Pipeline

    Data Engineer

    • Conducted daily stand meetings for task status discussions.
    • Applied technical knowledge in PySpark to solve day-to-day tasks.
    • Created DataFrames from different file formats.
    • Developed an end-to-end data pipeline for Retail stores data using S3, HDFS, Apache PySpark, MySQL, and PowerBi.
    • Installed required libraries in the cluster based on task requirements.
    • Utilized various transformations to process the data.
    • Wrote PySpark logic using PyCharm IDEA and Databricks.
    • Developed data extraction logic for the application.
    • Stored the processed data in required destinations using various file formats.
    • Ensured efficient and optimized code implementation.
    • AWS S3
    • AWS EMR
    • AWS Glue
    • HDFS
    • MySQL
    • PowerBi
    View Details

    Migration of Oracle Database to Snowflake

    Data Engineer

    • Extracted, loaded, and transformed data using Snowflake utilities like SnowSQL.
    • Imported data from Oracle and stored it in AWS S3 bucket.
    • Developed Snowflake SQL queries and scripts for data analysis and cleansing.
    • Utilized Snowflake utilities such as SnowSQL, SnowPipe, Python, Tasks, Streams, Time travel, Optimizer, Metadata Manager, data sharing, and stored procedures.
    • Applied ETL concepts and data warehousing principles.
    • Familiarity with data visualization tools like Power BI.
    • Monitored job execution and resolved any issues that occurred.
    • Uploaded output data to the designated directory on S3.
    • Addressed Jira tickets raised by clients.
    • Aws S3 Bucket
    • SnowSQL
    • SnowSQL
    • SnowPipe
    • Python
    • Tasks
    • Streams
    • SQL
    • Power BI
    View Details

    Sentiments Analysis Application Development And Deployment

    Data Engineer

    • Participated in daily scrum calls with team members to discuss tasks and progress.
    • Automated data processing pipeline using Sagemaker.
    • Developed new features and modifications as per requirements.
    • Conducted weekly calls with clients to understand their requirements and plan enhancements.
    • Utilized Step Function, Lambda, and code.py in Docker image stored in ECR for data processing.
    • Created a Step Function to access the Docker image and trigger it from AWS Lambda function.
    • Developed AWS services including API, RDS instances, and Lambda for serverless applications.
    • Collaborated with IT team and other departments for web application migration to AWS.
    • Managed tight timelines when required.
    • Supported teammates in importing data from source to S3 using AWS Sagemaker.
    • Imported data from S3 raw bucket, performed cleaning and masking, and moved it to S3 cleanse bucket using Glue.
    • Created DDL scripts for table creation in Redshift.
    • Validated records after transformation and loading of data into Redshift using Glue job.
    • AWS
    • S3
    • Lambda
    • CloudWatch
    • Glue
    • Sagemaker
    • ECR
    • Docker image
    • code.py
    View Details

    Education

    B

    MBA

    B.A.M.U. University
    2012 - 2014
    B

    B.Sc. Computer Science

    B.A.M.U. University
    2008 - 2011