Senior PySpark Developer / Senior Data Engineer

Position: PySpark Developer / Senior Data Engineer

Experience: 10+ Years (Relevant Experience Mandatory)

Location: Hyderabad | Bangalore | Pune | Chennai

Employment Type: Contract to Hire (C2H)

Work Mode: Hybrid

Working Days: 5 Days/Week

Shift Timings: 11:00 AM – 8:00 PM IST

Interview Process: 3 Rounds

Notice Period: 0–15 Days

Job Summary

We are seeking an experienced PySpark Developer / Senior Data Engineer with strong expertise in PySpark, Python, SQL, and Google BigQuery. The ideal candidate will design, develop, and optimize scalable ETL/ELT pipelines while ensuring high-performance data processing and quality in a cloud-based environment.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using PySpark.
• Build and optimize Spark-based data processing workflows for large datasets.
• Develop robust data transformation logic using Python and SQL.
• Work with Google BigQuery for data warehousing and analytics.
• Optimize data pipelines for performance, scalability, and reliability.
• Implement data quality, governance, and observability best practices.
• Collaborate with cross-functional teams in an Agile environment.
• Troubleshoot, debug, and optimize existing data engineering solutions.
• Participate in code reviews and contribute to engineering best practices.

Required Skills

• Strong hands-on experience with PySpark, Python, and SQL.
• Experience designing and optimizing Spark-based ETL/ELT pipelines.
• Strong understanding of Google BigQuery.
• Knowledge of data quality, governance, observability, and performance tuning.
• Strong analytical, debugging, and problem-solving skills.
• Experience working in Agile/Scrum teams.

Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
• 6+ years of Data Engineering experience with strong hands-on expertise in PySpark.