Data Engineer- AWS / Azure

Azure Data Engineer

Role Description

We are seeking a skilled and motivated Senior Data Engineer to join our engineering team. He/she will be responsible for building robust and scalable Data pipelines while collaborating across teams to create scalable, high-quality, maintainable product.

Technical Skills

• Python
• PySpark
• Databricks
• Azure Cloud platform
• NoSQL Databases (cosmosdb, mongodb, …)
• UnityCatalog
• Datascience basic knowledge

Responsibilities

• Design, build, and maintain data transformation pipelines using PySpark and Databricks.
• Manage effeciently large dataset.
• Manage NoSQL database - CosmosDB.
• Implement and maintain complex data calculation formula.
• Open to collaborate with Software Engineers.
• Participate in code reviews, design discussions, and software planning.
• Contribute to automation, testing, and deployment processes.

Experience- 7 – 13 years

Location: Pune, Bangalore, Delhi

AWS Data Engineer

Role Description

We are seeking a skilled and motivated Senior Data Engineer to join our engineering team. He/she will be responsible for building robust and scalable Data pipelines while collaborating across teams to create scalable, high-quality, maintainable product.

Technical Skills

• Proficiency in Python for data transformation and automation.
• Experience with Terraform and AWS Cloud infrastructure.
• Familiarity with Java for SCADA software development.
• Understanding of Data Lake architecture and data modeling.
• Knowledge of MQTT, Greengrass, and OPC-UA protocols.
• Experience with Windows Server management and PowerShell scripting.
• Ability to create and manage Grafana dashboards using SQL queries.
• Basic knowledge of AWS CloudWatch and AWS Glue.

Role Description

• Design and implement data transformation workflows using Python and AWS services.
• Develop and maintain infrastructure using Terraform and AWS Cloud.
• Standardize and manage raw and curated data in a multi-source Data Lake.
• Build and maintain SCADA data collection software deployed on AWS Greengrass.
• Automate deployment of Greengrass across multiple sites and ensure operational stability.
• Monitor and manage Windows servers hosting Greengrass using PowerShell.
• Create and maintain Grafana dashboards for data visualization and system monitoring.
• Coordinate with IT teams to manage incidents and plan maintenance activities.
• Use AWS CloudWatch to monitor system health and report errors.
• Integrate CRM data (Salesforce) into Power BI using AWS Glue and Python Lambdas.
• Collaborate with Data Platform teams to manage access, API provisioning, and folder creation.

Highly Recommended

• Advanced understanding of SCADA systems and industrial data flows.
• Experience with automated deployment and multi-site configuration management.
• Familiarity with network administration and cybersecurity principles.
• Basic knowledge Salesforce integration.
• Optional
• Technical Skills:
• Experience with Power BI dashboard creation.
• Knowledge of API management and data access provisioning on enterprise platforms.

Experience- 7 – 13 years

Location: Pune, Bangalore, Delhi