Senior Software Engineer – Data Ingest Engineering(Kafka)

What You’ll Do

• Design and implement high-throughput, event-driven architectures to power near real-time data pipelines for ingesting into Azure + Databricks Lakehouse
• Build cloud-native data pipelines to support batch ingest workloads
• Configure Unity Catalog by designing Bronze/Silver/Gold medallion architecture and optimize for centralized data governance, fine-grained access control, and end-to-end lineage tracking across Databricks workspaces
• Drive engineering excellence by setting standardized DevOps practices and optimizing data engineering code for performance
• Identify and design solutions to potential data processes that can be improved with automation
• Partner with domain teams to onboard new capability use cases, mentor engineers, and contribute to reusable blueprints and reference implementations.

What You’ll Bring

• 7+ years of software engineering experience focused on data infrastructure and backend systems.
• Hands-on experience with the Microsoft Azure data platform — Azure Data Factory , Event Hubs, Blob Storage, ADLS Gen2, and Azure DevOps for orchestration and CI/CD
• Production experience with Azure Databricks and Delta Lake
• Production experience with PySpark for distributed data processing and transformation.
• Streaming frameworks (Kafka or comparable) for real-time data paths including CDC and incremental batch patterns.
• Strong communication skills with the ability to influence senior stakeholders and lead technical decisions across global teams.

Must Have Skills

• Data Platform Architecture (Lakehouse, Delta Lake, Azure Databricks)
• Kafka / Streaming Frameworks
• Strong SQL and Python Proficiency
• Experience with Cloud Native Tools

Nice-to-Have

• Building data infrastructure for AI use cases
• Prior experience in large-enterprise data integration or platform engineering environments
• Prior experience in enterprise platform migrations