Senior Manager Data Engineering - Big Data

Job DescriptionPeople Data Platform – Powering Secure, Enterprise-Scale People Insights at Walmart .The People Data Platform team at Walmart owns the centralized People Data Lake that enables trusted, cross-domain people insights across the enterprise. We build and operate a scalable data platform on GCP, ingesting data from multiple source systems and transforming it through a robust medallion architecture to deliver high-quality, analytics-ready data.

Our team designs data and semantic models that empower HR, business, and analytics teams to make data-driven decisions at scale. In parallel, we lead the Data Security charter for the People Data Lake, implementing encryption, masking, and fine-grained access controls to protect sensitive data while ensuring compliant and governed access.

The Workforce Intelligence Platform (WIP) builds the access and experience layer on top of this foundation-enabling secure, personalized, and scalable analytics consumption. WIP combines query execution, dynamic access control (RLS, CLS, ABAC), and API-driven architecture to power dashboards, applications, and AI-driven data interactions.

Operating at the intersection of big data, cloud, security, and platform engineering, we are building a future-ready analytics platform that supports one of the world's largest workforces.

Lead and grow a high-performing team of 8-10 data engineers building and operating the People Data Lake

Own the architecture, design, and delivery of scalable batch and future streaming data pipelines using Spark, Big Query, Airflow, and related cloud-native technologies.

Drive best practices across data modeling, pipeline reliability, observability, and cost optimization to ensure high performance and availability.

Define and implement the data security framework for the lake, including encryption (at rest and in transit), masking and fine-grained access controls.

Establish secure data sharing and governance mechanisms to protect sensitive data and ensure regulatory and enterprise compliance.

Partner with Product, HR, Analytics, Security, and Architecture teams to define roadmap, prioritize initiatives, and deliver scalable, resilient, and secure data solutions.

Build a strong engineering culture focused on operational excellence, technical rigor, and continuous improvement while shaping the mid- to long-term platform vision.

Establish best engineering and operational excellence practices based on product, engineering, and scrum metrics.

Participate in hiring, mentoring, and building high performing agile teams.

Establishing clear expectations with individuals based on their level and role, meeting regularly to discuss performance, provide feedback and coaching

12+ years of experience in software or data engineering with 3+ years leading high-performing engineering teams.

Strong hands-on expertise in Spark, BigQuery, Airflow (or equivalent), and distributed data systems on GCP.

Experience with big data and distributed systems like Spark, Flink, etc.

Proven experience designing, building, and operating large-scale data lakes or lakehouse platforms with batch and streaming ingestion.

Deep understanding of data security principles including encryption, PII protection, masking, IAM, and row/column-level security.

Strong SQL and data modeling skills with experience implementing CI/CD, monitoring, and reliability best practices for data platforms.

Knowledge of cloud platforms like Azure, GCP etc

Lead the integration of AI/ML toolkits and frameworks into day-to-day (D2D) data engineering workflows, enabling teams to accelerate development, automate data processes, and enhance decision-making capabilities.

Thats what we do at Walmart Global Tech. Were a team of software engineers, data scientists, cybersecurity experts and service professionals within the worlds leading retailer who make an epic impact and are at the forefront of the next retail disruption. People are why we innovate, and people power our innovations. We are people-led and tech-empowered.

We train our team in the skillsets of the future and bring in experts like you to help us grow. Here, you can kickstart a great career in tech, gain new skills and experience for virtually every industry, or leverage your expertise to innovate at scale, impact millions and reimagine the future of retail.

Walmart's culture sets us apart, and we know being together helps us innovate, learn and grow great careers. This role is based in our [Bangalore/Chennai] office for daily work, with the flexibility for associates to manage their personal lives.

Beyond our great compensation package, you can receive incentive awards for your performance. Other great perks include a host of best-in-class benefits maternity and parental leave, PTO, health benefits, and much more.

By fostering a workplace culture where everyone isand feelsincluded, everyone wins. By making Walmart a welcoming place where all people feel like they belong, were able to engage associates, strengthen our business, improve our ability to serve customers, and support the communities where we operate.

Walmart, Inc. is an Equal Opportunity Employer By Choice. We believe we are best equipped to help our associates, customers and the communities we serve live better when we really know them. That means understanding, respecting, and valuing unique styles, experiences, identities, ideas and opinions while being welcoming of all people.

Minimum Qualifications:Option 1: Bachelor's degree in Computer Science and 5 years' experience in software engineering or related field. Option 2: 7 years' experience in software engineering or related field. Option 3: Master's degree in Computer Science and 3 years' experience in software engineering or related field. 4 years' experience in data engineering, database engineering, business intelligence, or business analytics. 1 year's supervisory experience.

Tower 1, Part Of 1st, 2nd To 4th Flrs, Intl Tech Park Radial Road (radial It Park Pvt Ltd), , India