Sr Devops Engineer with Databricks

• 6+ years in data platform engineering, DataOps, platform engineering, DevOps or SRE.
• At least 5+ years of hands-on Terraform and IaC experience, including modules, state management, imports, plans, provider and version control, environment separation, drift detection and CI/CD.
• Strong Python engineering for APIs, automation, testing, packaging, structured logging, error handling and reusable tooling.
• Production experience operating Databricks and Unity Catalog.
• Strong production Apache Airflow experience.
• Experience operating dbt environments, jobs and CI/CD workflows.
• AWS IAM, S3, VPC/private connectivity, KMS, Secrets Manager and CloudWatch.
• Git, CI/CD, REST APIs, SQL, Linux, OAuth, SAML/SCIM and service-principal authentication.
• Experience with monitoring, incident response, change control, rollback and cost management.
• Relevant Databricks platform training or certification is highly preferred.

;

Responsibilities

• Provision and manage platform resources through reusable Terraform modules and configuration as code.
• Operate Databricks accounts and workspaces, including identities, groups, service principals, entitlements, compute policies, clusters, SQL warehouses and Jobs/Workflows.
• Configure Unity Catalog metastores, catalogs, schemas, external locations, storage credentials, workspace bindings, ownership and least-privilege access.
• Operate dbt projects, environments, connections, jobs, permissions, service tokens, CI quality gates and promotion controls.
• Engineer production Astro/Airflow capabilities covering DAG standards, dependencies, retries, timeouts, concurrency, backfills, ing and safe recovery.
• Orchestrate and integrate Fivetran, dbt, Databricks and Atlan using APIs, webhooks and workload identities.
• Implement Fivetran usage monitoring, cost thresholds, non-production lifecycle guardrails and configuration portability.
• Enable governed Atlan and Tableau connectivity, metadata, lineage, access and operational workflows.
• Implement ADLC controls for branching, testing, approval, environment promotion, deployment evidence and rollback.
• Integrate identity and access workflows with Okta or equivalent identity providers and ServiceNow.
• Establish platform and pipeline SLOs; monitor reliability, schema changes, data quality, usage and cost.
• Troubleshoot incidents, perform root-cause analysis and automate recurrence prevention.
• Maintain runbooks, configuration inventories, access evidence and recovery procedures.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.