Data Engineer- AWS / Azure

Azure Data Engineer

Role Description

We are seeking a skilled and motivated Senior Data Engineer to join our engineering team. He/she will be responsible for building robust and scalable Data pipelines while collaborating across teams to create scalable, high-quality, maintainable product.

Technical Skills

PythonPySparkDatabricksAzure Cloud platformNoSQL Databases (cosmosdb, mongodb, )UnityCatalogDatascience basic knowledge

Responsibilities

Design, build, and maintain data transformation pipelines using PySpark and Databricks. Manage effeciently large dataset.Manage NoSQL database - CosmosDB.Implement and maintain complex data calculation formula.Open to collaborate with Software Engineers.Participate in code reviews, design discussions, and software planning. Contribute to automation, testing, and deployment processes.

Experience- 7 13 years

Location: Pune, Bangalore, Delhi

AWS Data Engineer

Role Description

We are seeking a skilled and motivated Senior Data Engineer to join our engineering team. He/she will be responsible for building robust and scalable Data pipelines while collaborating across teams to create scalable, high-quality, maintainable product.

Technical Skills

Proficiency in Python for data transformation and automation.Experience with Terraform and AWS Cloud infrastructure.Familiarity with Java for SCADA software development.Understanding of Data Lake architecture and data modeling.Knowledge of MQTT, Greengrass, and OPC-UA protocols.Experience with Windows Server management and PowerShell scripting.Ability to create and manage Grafana dashboards using SQL queries.Basic knowledge of AWS CloudWatch and AWS Glue.

Role Description

Design and implement data transformation workflows using Python and AWS services.Develop and maintain infrastructure using Terraform and AWS Cloud.Standardize and manage raw and curated data in a multi-source Data Lake.Build and maintain SCADA data collection software deployed on AWS Greengrass.Automate deployment of Greengrass across multiple sites and ensure operational stability.Monitor and manage Windows servers hosting Greengrass using PowerShell.Create and maintain Grafana dashboards for data visualization and system monitoring.Coordinate with IT teams to manage incidents and plan maintenance activities.Use AWS CloudWatch to monitor system health and report errors.Integrate CRM data (Salesforce) into Power BI using AWS Glue and Python Lambdas.Collaborate with Data Platform teams to manage access, API provisioning, and folder creation.

Highly Recommended

Advanced understanding of SCADA systems and industrial data flows.Experience with automated deployment and multi-site configuration management.Familiarity with network administration and cybersecurity principles.Basic knowledge Salesforce integration.OptionalTechnical Skills:Experience with Power BI dashboard creation.Knowledge of API management and data access provisioning on enterprise platforms.

Experience- 7 13 years

Location: Pune, Bangalore, Delhi .