Data Engineering Specialist
Quest Global
Pune, Maharashtra, India
Full-Time
Posted 12 days ago
Pune, Maharashtra, India
Full-Time
Posted 12 days ago
Job Description & Responsibilities
Job Requirements
- · Design and develop robust data ingestion and ETL pipelines for batch and workflow-based processing requirements.
- · Build, optimize, and maintain data processing solutions using Apache Spark, PySpark, and Apache NiFi.
- · Develop, schedule, monitor, and manage workflows in Apache Airflow to ensure reliable orchestration.
- · Work on the Cloudera Data Engineering platform and the broader Hadoop ecosystem, including HDFS and Hive.
- · Write efficient Hive SQL and Python code for data transformation, validation, and operational automation.
- · Support, troubleshoot, and optimize production data pipelines for performance, reliability, and scalability.
- · Collaborate with data architects, analysts, platform teams, and business stakeholders to understand requirements and deliver data solutions.
- · Follow engineering best practices for code quality, version control, documentation, deployment, and monitoring.
Work Experience
· 8+ years of relevant experience in data engineering, big data platforms, and production pipeline delivery.
· Strong hands-on experience in designing data pipelines using NiFi, Spark/PySpark, Hive SQL, and Python.
· Good understanding of Hadoop concepts, HDFS storage patterns, data partitioning, and distributed processing.
· Experience managing Airflow DAGs, dependencies, retries, monitoring, and failure handling.
· Ability to identify pipeline bottlenecks and perform performance tuning across ingestion, transformation, and orchestration layers.
Strong problem-solving mindset with the ability to support critical production workloads.
About Quest Global
Required Skills
PythonSQL
Job Details
Employment TypeFull-Time
Work ModeOn-Site
Experience8 – 13 years
Positions1
Posted by
N/A
Posted on:
6 Nov 2026
About Quest Global
More open roles
- Senior Software Engineer (Python/Node.JS , AWS)Autodesk · India
- IT Intern - Software Development, Python & AI/MLAI SAMRAT · India
- Ansible & Python Open-source Developer for GamingHartbridge Evergrove Placements · Farrukhabad, Uttar Pradesh, India
- Senior Full Stack Developer (Python, React.js & Azure, AI)Robert Bosch · India
- Specialist System Engineering (Devops Engineer, CICD, Kubernetes, containers, React, Python, API, Deployment + Development + Automation, GitHub PipelinesAT&T · Bengaluru, Karnataka, India
- Data Scientist (Python, ML, SQL) (Bhilai)Workers Town · Raipur, Chhattisgarh, India