Lead Cloud Data Engineer - Contract (India)

Role : Lead Cloud Data Engineer

Contract : 3-6 Months

Shift - 12 PM IST - 9 PM IST

Location - REMOTE

Role Overview

We are seeking a Lead Cloud Platform Engineer with deep expertise in AWS infrastructure, Databricks platform administration, CI/CD automation, and observability tooling. This individual will serve as a technical leader responsible for designing, governing, and optimizing cloud and data platform environments while mentoring engineers and driving platform reliability, scalability, and operational excellence.

Key Responsibilities

Technical Leadership

• Lead the design, implementation, and support of enterprise-scale AWS and Databricks platforms.
• Provide technical guidance and mentorship to cloud engineers, platform engineers, and DevOps teams.
• Define platform architecture standards, best practices, and operational procedures.
• Act as the primary escalation point for complex infrastructure and platform-related issues. AWS Infrastructure & Cloud Operations
• Architect and manage AWS environments, including networking, storage, compute, security, and monitoring services.
• Drive cloud governance, cost optimization, security compliance, and infrastructure standardization initiatives.
• Review and optimize cloud resource utilization and performance. Databricks Platform Management
• Lead Databricks workspace administration, configuration, and platform optimization efforts.
• Oversee cluster strategies, job orchestration, workspace governance, and environment management.
• Implement and manage Unity Catalog, access controls, and security policies.
• Partner with Data Engineering teams to ensure platform scalability and operational efficiency.

DevOps & CI/CD

• Design and implement enterprise CI/CD frameworks and deployment strategies.
• Drive Infrastructure as Code (Terraform) adoption across environments.
• Establish automated deployment, testing, and release management processes.
• Promote DevSecOps best practices throughout the software delivery lifecycle.

Monitoring & Reliability Engineering

• Define and implement observability and monitoring strategies across AWS and Databricks environments.
• Establish SLAs, SLOs, monitoring dashboards, alerting mechanisms, and operational metrics.
• Lead incident response, root cause analysis, and continuous improvement initiatives.
• Improve platform reliability, availability, and performance through proactive monitoring.

Required Skills

Cloud Technologies

• AWS (EC2, VPC, IAM, S3, EKS, Lambda, RDS, CloudWatch, Secrets Manager)
• Cloud Security and Governance, Networking and Infrastructure Architecture Databricks
• Databricks Administration
• Cluster Management and Optimization
• Unity Catalog
• Workspace Governance
• Job Orchestration and Scheduling
• Performance Tuning and Troubleshooting

DevOps & Automation

Terraform, GitHub Actions / Azure DevOps / Jenkins / GitLab CI/CD, Git Version Control, Infrastructure as Code (IaC), Python and Shell Scripting

Monitoring & Observability

• Datadog, Grafana, Splunk, New Relic, CloudWatch
• Logging, Alerting, and Incident Management

Required Experience

• 10+ years of overall IT experience.
• 5+ years of cloud platform engineering experience in AWS.
• 3+ years of Databricks administration and platform support.
• Proven experience leading engineering teams and platform transformation initiatives.
• Experience supporting enterprise data and analytics platforms.

Pay: ₹80,000.00 - ₹100,000.00 per month

Perks

• Work from home

Work Location: Remote