Senior Pyspark Data Engineer (With Fabrics)

Job Title: Senior PySpark Data Engineer (5+ Years Experience)

Location: Remote / Hybrid

Experience Level: 5+ Years

Role Overview

We are seeking an experienced Data Engineer to design, build, and optimize our large-scale data pipelines and analytics architecture. You will work extensively with cloud data platforms to transform complex datasets into actionable business insights.

Mandatory Skills

• PySpark & Python: 5+ years of hands-on experience building distributed data processing pipelines and advanced data transformations.
• Databricks: Strong expertise in managing Databricks workspaces, Delta Lake, and optimizing cluster performance.
• Azure Data Factory (ADF): Proven track record in orchestrating ETL/ELT workflows and pipeline automation.
• SQL: Advanced query optimization, database design, and data modeling skills.

Preferred Skills

• Hands-on experience with Microsoft Fabric (OneLake, Synapse Data Engineering, Real-Time Analytics).

Key Responsibilities

• Architect, develop, and maintain robust ETL/ELT pipelines using PySpark and ADF.
• Optimize Databricks jobs and SQL queries for performance and cost efficiency.
• Collaborate with data science and analytics teams to support data delivery requirements.
• Enforce data governance, quality standards, and security best practices across all data assets.

Pay: Up to ₹2,000,000.00 per year

Benefits

• Flexible schedule

Experience

• Pyspark: 5 years (Required)
• Python: 5 years (Required)
• Databricks: 5 years (Required)
• Azure: 5 years (Required)
• Fabrics: 1 year (Preferred)

Work Location: Remote