Big Data Engineer (Hyderabad)

Job Information- Date Opened 07/23/2025 - Industry Information Technology - Job Type Full time - Work Experience 5+ years - City Hyderabad - State/Province Telangana - Country India - Zip/Postal Code 500039 Job DescriptionCore Responsibilities- Design and optimize batch/streaming data pipelines using Scala, Spark, and Kafka - Implement real-time tokenization/cleansing microservices in Java - Manage production workflows via Apache Airflow (batch scheduling) - Conduct root-cause analysis of data incidents using Spark/Dynatrace logs - Monitor EMR clusters and optimize performance via YARN/Dynatrace metrics - Ensure data security through HashiCorp Vault (Transform Secrets Engine) - Validate data integrity and configure alerting systemsRequirements- Programming :Scala (Spark batch/streaming), Java (real-time microservices) - Big Data Systems: Apache Spark, EMR, HDFS, YARN resource management - Cloud & Storage :Amazon S3, EKS - Security: HashiCorp Vault, tokenization vs. encryption (FPE) - Orchestration :Apache Airflow (batch scheduling) - Operational Excellence Spark log analysis, Dynatrace monitoring, incident handling, data validationMandatory Competencies- Expertise in distributed data processing (Spark on EMR/Hadoop) - Proficiency in shell scripting and YARN job management - Ability to implement format-preserving encryption (tokenization solutions) - Experience with production troubleshooting (executor logs, metrics, RCA)BenefitsAdvantagesInsurance - Family Term Insurance PFPaid Time Off - 20 days Holidays - 10 days Flexi timing Competitive Salary Diverse & Inclusive workspace