Still Struggling with Tech Hiring? Discover Olibr's Solution Now!
Login / RegisterDeveloperClient
Login/Register

Rajendra Patel

Data Engineer

  • Bangalore Urban, KA
  • 2 years

Available From: Available

    Candidate's About

    I am a data engineer with 2+ years of experience in technologies such as Spark, ETL Pipelines, Data Warehouse, Airflow, Cassandra, MongoDB, and MySQL. I have worked in the manufacturing and finance industries and have developed efficient ETL pipelines to process over 1 TB of daily data, providing a comprehensive view of customer activities from product usage to back-office transactions.

    Work & Experience

    S

    Specialist Programmer (Data Engineer)

    Sep 20 - PRESENT

    Technology

    • Icon
      Apache Spark
    • Icon
      Python
    • Icon
      SQL (Advance)
    • Icon
      Problem Solving
    • Icon
      AWS
    • Icon
      Hadoop
    • Icon
      Excel

    Tools

    • Icon
      Github
    • Developed ETL pipelines for comprehensive customer data management
    • Created automation script to enhance the ETL process and reduce the workload on the development team
    • Involved in building system design and architecture of a data lake for finance clients
    • Created data pipeline using App flow to load data to third-party SaaS Applications
    • Automation script help...
    Read More

    Global Foundries

    Specialist Programmer (Data Engineer)

    • Developed ETL pipelines for processing 1 TB of daily data for a full view of customers
    • Transformed 50 ETL tables with complex logic, completing 40% of milestone work
    • Conducted end-to-end data validation for 100 tables
    • Developed enhanced automation script, reducing workload for the development team by 25%
    • Owned and tested UAT environment for 100 tables
    • Successfully deployed 3 milestones in production
    • Worked with various file formats such as parquet, CSV, tab, and zip
    • Pyspark
    • Python
    • SQL
    • AWS-EMR
    • AWS-Lambda
    • AWS-S3
    • AWS- Step Function
    • AWS-DynamoDb
    • Boto3
    • Airflow
    View Details

    Fittle (Xerox)

    Specialist Programmer (Data Engineer)

    • Designed and developed the data lake architecture for payments, credits, and decision making
    • Created data pipelines to load data to third-party SaaS applications such as Salesforce using AWS Appflow
    • Developed lambda functions to handle streaming data and offload it to AWS RDS
    • Developed automation script for building CloudFormation stack and data validation, reducing workload for the development team by 25%
    • Extracted data from various sources such as Oracle and SQL Server and ingested it into AWS S3 bucket
    • Created views using PySpark in AWS Glue according to business logic
    • Pyspark
    • AWS-GLue
    • AWS-Lambda
    • AWS-Redshift
    • AWS-RDS
    • AWS - Step Function
    • AWS-SNS
    View Details

    Education

    C

    B.Tech Computer Engineering

    Chandubhai S. Patel Institute of Technology
    2016 - 2020

    Certificates

    M

    Azure Data Fundamentals

    Microsoft
    N

    Designing analysis and algorithms

    National Program on Technology Enhanced Learning
    M

    Azure Fundamentals

    Microsoft
    N

    Programming, data structures and algorithms using Python | NPTEL

    National Program on Technology Enhanced Learning (NPTEL)