Data Engineer - Scala / Spark / Kafka

Role: Data Engineer – Scala / Spark / Kafka

Location

India

Employment Type

Contract

Experience

5+ Years

Duration

6–12 Months

Start Date

ASAP

Job Description

We are looking for an experienced

Data Engineer

to design, build, and optimize scalable data platforms capable of processing large volumes of batch and real-time data.

The ideal candidate will have strong hands-on experience with

Scala, Apache Spark, Kafka, and Big Data technologies . You will be responsible for developing robust data pipelines, designing scalable data models, troubleshooting performance issues, and collaborating with engineering and analytics teams to ensure reliable access to high-quality data.

The role involves extensive work with

distributed processing technologies, streaming platforms, and modern cloud-based data infrastructure .

Key Responsibilities

Design, develop, and maintain scalable data pipelines for batch and real-time processing.

Develop data processing applications using

Scala and Apache Spark .

Build and maintain real-time streaming pipelines using

Kafka and Spark Structured Streaming .

Design and implement scalable data models and data processing solutions.

Work with

Hive, SQL, S3, Presto, Delta Lake , and other Big Data technologies.

Analyze and optimize Spark jobs and distributed data workloads.

Troubleshoot data pipeline, performance, and production issues.

Ensure data quality, reliability, scalability, and availability.

Collaborate with engineering, analytics, and other technical teams to deliver data solutions.

Contribute to the design and development of modern data lake and lakehouse architectures.

Required Skills

5+ years of professional experience

in Data Engineering, Big Data Engineering, or a related field.

Strong programming skills in

Scala .

Hands-on experience with

Apache Spark

for both batch and streaming workloads.

Strong proficiency in

Hive and SQL .

Hands-on experience with

Kafka

and real-time event streaming.

Strong understanding of Big Data technologies and ecosystems.

Experience with

S3, Hive, Presto, and Delta Lake .

Experience designing and implementing scalable data models.

Strong debugging and problem-solving skills.

Experience with performance analysis and optimization of distributed data workloads.

Bachelor's or Master's degree in Computer Science, Engineering, or a related field.