GCP Kafka Integration Engineer

Senior Data Engineer — Kafka & GCP (Event-Driven Platform) | US Hours (Shift)

Location: India · Workplace type: 100% in-Office

Build a mission-critical event platform from the ground up — and own it.

We're replacing our Legacy enterprise system with a modern event-driven architecture on Google Cloud, and we're hiring the engineer who will design and build it: a managed Apache Kafka backbone processing ~10 million events per day, real-time rule processing, and a compliant 7-year event archive on BigQuery and Cloud Storage.

This is a rare greenfield: no legacy cloud code to inherit, real scale from day one, and decisions that are yours to make and defend. You'll partner directly with a senior legacy functional expert who owns everything on the Legacy side. You bring deep Kafka, GCP, and Python engineering; we bring the business and legacy domain knowledge and a clear mandate to ship.

What you'll build

• A Google Cloud Managed Service for Apache Kafka cluster: topics, partitioning strategy, security, retention
• A Python event-ingestion service on Cloud Run receiving events from Legacy and external logistics carriers
• A Kafka consumer rule engine maintaining live state in Firestore and triggering actions back into external system
• Kafka Connect pipelines archiving every event to BigQuery and to a locked, audit-grade Cloud Storage vault
• The monitoring, dead-letter handling, and reconciliation that make "zero events lost" a provable fact, not a hope

What you bring

• 5+ years in backend or data platform engineering
• 2+ years running Apache Kafka in production (any distribution — open source, Confluent, MSK, Aiven): you can explain partitions, keys, consumer groups, offsets, and delivery semantics from lived experience
• 2+ years hands-on GCP: Cloud Run or GKE, IAM and service accounts, VPC fundamentals
• Strong production Python: services, not just scripts
• BigQuery fluency, including partitioned and clustered table design
• A reliability mindset: idempotency, retries, dead-letter queues, and monitoring are second nature
• End-to-end ownership: you design it, build it, run it, and answer for it

Bonus points (Not Mandatory)

• Google Cloud Managed Kafka or Kafka Connect specifically
• Firestore, Bigtable, or similar NoSQL at scale
• Terraform / IaC
• Regulated data retention (WORM / audit) experience

Working hours — please read before applying

This role runs on a fixed shift of 6:30 PM – 3:30 AM IST, Monday–Friday, fully aligned with US business hours. It is 100% in-office from Kalyan, Maharashtra, India, with a night-shift allowance paid on top of compensation. We state this upfront because this role only works for people whose lives genuinely fit these hours.

Our process (fast and hands-on)

• 30-minute technical + fit call
• 2-hour practical round in a live GCP sandbox — real Kafka, real BigQuery, a real debugging task
• 1-hour design discussion Decision within 2 working days of your final round. We respect your time.