Home/Job List/Senior Databricks Architect (10–15+ years)
Zorba AI

Senior Databricks Architect (10–15+ years)

Zorba AI

India
Full-Time
Posted 12 days ago

Job Description & Responsibilities

Senior Databricks Architect (10–15 years)

Role Overview

Senior platform and solution architect responsible for enterprise-scale Databricks lakehouse architecture,

modernization strategies, governance, migration roadmaps, engineering standards, performance and cost

guardrails, and AI-ready data foundations. The role converts business and non-functional requirements into

implementable platform patterns and guides engineering teams through delivery.

Core Responsibilities

Lakehouse & Platform Architecture

 Define target-state lakehouse architecture, workspace and environment strategy, Unity Catalog topology,

storage and compute patterns, governed sharing, lineage and domain boundaries.

 Establish implementation standards for Medallion architecture, Lakeflow Declarative Pipelines / DLT,

Workflows, Auto Loader, Delta Lake, system tables and Databricks Asset Bundles.

 Create reference architectures, architecture decision records, reusable blueprints and non-functional

requirement guardrails.

Spark, Data Engineering & Streaming

 Set engineering standards for PySpark, Spark SQL, Delta Lake, Structured Streaming, CDC, SCD,

incremental processing, data contracts and schema evolution.

 Guide execution-plan analysis, AQE, join strategy, shuffle and skew management, partitioning, file sizing,

caching and Photon adoption.

 Drive idempotent, auditable and recoverable pipelines with clear batch-versus-streaming decision rationale.

Migration & Modernization

 Lead legacy warehouse, Hadoop and ETL modernization assessments and define migration waves,

conversion patterns and dependency plans.

 Own validation, reconciliation, parallel-run, cutover, rollback and decommissioning approaches.

 Create reusable migration playbooks and accelerators while controlling delivery risk and business continuity.

Governance, Security & Data Sharing

 Design Unity Catalog operating patterns covering catalogs, schemas, external locations, service principals,

RBAC/ABAC, masking, row/column controls and auditability.

 Define lineage, data quality, privacy, retention and compliance-ready implementation standards.

 Guide Delta Sharing, Lakehouse Federation and governed producer-consumer patterns where appropriate.

Platform Engineering, DataOps & Reliability

 Define Git-based CI/CD, testing, environment promotion, infrastructure-as-code and release controls for

Databricks assets.

 Establish SLIs/SLOs, observability, alerting, runbooks, reliability scorecards and production-readiness

reviews using platform telemetry and system tables.

 Improve developer experience through templates, reusable frameworks, guardrails and self-service patterns.

Cost Optimization & FinOps

 Establish cluster and compute policies, autoscaling, workload sizing, serverless decision guidelines, tagging

and chargeback/showback controls.

 Create cost and performance baselines and drive workload-level optimization with measurable outcomes.

 Balance performance, reliability and cost rather than optimizing one dimension in isolation.

Databricks Hiring JD Pack v2

  • C5–C6

AI-ready Data Foundations

 Guide governed data foundations for RAG and AI use cases, including data quality, lineage, access controls

and retrieval readiness.

 Demonstrate working awareness of Vector Search, model serving/lifecycle, MLflow, context engineering and

AI governance.

 Assess emerging capabilities without weakening core data engineering and platform-accountability

expectations.

Consulting & Stakeholder Leadership

 Lead discovery and architecture workshops; articulate options, trade-offs, risks, roadmap and estimates to

technical and business stakeholders.

 Explain the latest project with clear business problem, architecture choice, Databricks fit, alternatives

considered, challenges, outcomes and lessons learned.

 Mentor engineers and leads, review designs, and support proposals, solution narratives and customer-facing

architecture discussions.

Skills: databricks,architect,data platform

Required Skills

SQLGitCI/CDLeadership

Job Details

Employment TypeFull-Time
Work ModeOn-Site
Experience10 – 15 years
Positions1

Posted by

N/A

Posted on:

6 Nov 2026

About Zorba AI

More open roles

Browse all jobs →