Still Struggling with Tech Hiring? Discover Olibr's Solution Now!
Login / RegisterDeveloperClient
Login/Register

Anuj Mahajan

Senior Technology Consultant

  • Bhopal, MP
  • 6 years

Available From: Available

    Candidate's About

    With 5 years of experience, I specialize in building robust big data pipelines, ingesting data from diverse sources like APIs, Mainframes, and RDBMS. Proficient in Hadoop ecosystem tools, AWS services, Python, SQL, and more, I enable efficient data engineering, streaming ingestion, and real-time/batch integration for valuable insights.

    Work & Experience

    S

    Senior Technology Consultant

    Dec 21 - PRESENT

    Technology

    • Icon
      Hadoop
    • Icon
      Apache Spark
    • Icon
      SQL (Advance)
    • Icon
      MongoDB
    • Icon
      Python
    • Icon
      AWS
    • Icon
      Kafka
    • Icon
      Linux

    Tools

    • Icon
      Github
    •  Provide strategic technology consulting services to clients, focusing on digital transformation and technology innovation.
    • Collaborate with client stakeholders to understand their business objectives and identify technology solutions to meet their needs.
    • Conduct technology assessments and evaluations to identify gaps, risks, and opportunities for improvement.
    • ...
    Read More

    Piramal

    Data Engineer-EY

    • Worked for Data Migration of on premise PostgreSQL and MongoDB and salesforce Databases to AWS in snowflake tables. Worked on building data lake. 
    • Worked on creating an end to end solution by creating a data pipeline for the streaming data from mongoDB and PostgreSQL to Snowflake and DynamoDB and exposing the relevant columns to the API using Lambda and API gateway. 
    • Worked on Pyspark EMR and Kinesis to create a streaming application. Have worked on Airflow to schedule batch jobs. Also created Data Quality Framework using python, snowflake and airflow to check the quality of key data elements.
    • Pyspark
    • SQL
    • MongoDB
    View Details

    American Express

    Data Engineer

    • Worked on Data migration from SAS to Hadoop ecosystem. Converted the SAS code to Pyspark code. 
    • Created a Framework to calculate Loss Charge volume dynamically for all the modules based on Pyspark based on the inputs supplied through the YAML file. 


    • Pyspark
    View Details
    D

    Data Engineer

    Jul 20 - Dec 21

    Technology

    • Icon
      Hadoop
    • Icon
      Python
    • Icon
      SQL
    • Icon
      Kafka
    • Icon
      Linux
    • Icon
      Apache Spark
    • Worked as part of the GDIA team, responsible for ingesting data from various sources into the Hadoop ecosystem for downstream teams.
    • Utilized data ingestion tools such as Attinuity (QLIK) to extract, transform, and load data from different sources into the Hadoop ecosystem.
    • Developed a Data Reconciliation framework using Pyspark to ensure the accuracy and integrity of data ...
    Read More
    S

    Software Engineer

    Mar 18 - Jul 20

    Technology

    • Icon
      Hadoop
    • Icon
      Apache Spark
    • Icon
      Python
    • Icon
      SQL
    • Icon
      Linux
    • Was Part of the data migration team which aimed at creating new data repository for Payment Ruled Database. It involves Ingestion of data from MySQL source (existing database) into Hadoop Ecosystem using Sqoop
    • Worked in data transformation team applying Business logic in HIVE tables using Spark SQL in Python language as per the client requirement.
    • Created Data Pipeline to i...
    Read More

    Education

    B

    BTech

    Bansal Institute of Science and Technology
    2013 - 2017