Senior Platform Data Engineer (Databricks)

The Role

Architect and operate large-scale distributed data platforms using Databricks and Spark.

What You Will Do

• Architect Databricks deployments
• Optimize Spark and Delta Lake workloads
• Implement Infrastructure as Code
• Lead incident response
• Drive performance standards

What You Will Have

• 6+ years of data or cloud engineering experience
• 2+ years operating Databricks in production
• 2+ years working with Spark
• 2+ years implementing Infrastructure as Code
• Production ownership experience
• Responsible for the end-to-end administration, support, and maintenance of enterprise data platforms, including Databricks and Cloudera (CDP/CDH).
• Manages Databricks workspaces, compute, Unity Catalog, security, storage integrations, and CI/CD, while supporting Cloudera services such as HDFS, YARN, Hive, Impala, Spark, Oozie, Ranger, Atlas, Kerberos, and TLS.
• Performs platform monitoring, incident troubleshooting, Spark job analysis, log investigation, disk space management, Linux-based operational support (services, processes, file systems, permissions, and logs), performance tuning, upgrades, automation (Ansible/chef/Terraform), capacity planning, and ensures high availability, security, governance, and operational excellence across AWS, Azure, GCP and on-perm Linux environments.
• On-Call Support

What Will Set You Apart

• Certifications: Databricks, Cloudera, AWS
• Streaming architecture experience
• AI/ML workload enablement
• FinOps cost optimization expertise