Big Data Devops Engineer

Job Summary

We are looking for a Big Data DevOps Engineer to manage and optimize big data platforms, automate deployments, and ensure high availability and performance of data systems. The role bridges data engineering and DevOps practices to support scalable data pipelines and infrastructure.

Key Responsibilities

• Design, deploy, and manage big data infrastructure (Hadoop, Spark, Kafka, etc.).
• Build and maintain CI/CD pipelines for data applications and workflows.
• Automate infrastructure provisioning using tools like Terraform/CloudFormation.
• Monitor system performance, troubleshoot issues, and ensure high availability.
• Collaborate with data engineers and developers for seamless data pipeline deployment.
• Implement security, data governance, and compliance best practices.
• Optimize cluster performance, storage, and compute resources.
• Manage cloud-based data platforms (AWS, Azure, or GCP).