Data Engineer - Real Time Streaming Kafka

Experience

Minimum 5+ years of development and design experience in Java/Scala with Flink (Beam or Spark streaming) and Kafka.

Technical Skills

• Extensive coding experience and knowledge in Event driven and streaming architecture
• Experience in JVM tuning for performance
• Knowledge on Containerization using Docker and Kubernetes.
• Working knowledge of Caching systems, with experience using Redis will be preferable. [Nice to have]
• Linux OS configuration and use, including shell scripting.
• Good hands-on experience with design patterns and their implementation.
• Well versed with CI/CD principles (GitHub, Jenkins etc.), and actively involved in solving troubleshooting issues in distributed services ecosystem
• Experience working with SQL and NoSQL databases
• Familiar with Distributed services resiliency and monitoring in a production environment.
• Experience in designing, building, testing and implementing security systems including identifying security design gaps in existing and proposed architectures and recommend changes or enhancements.
• Responsible for adhering to established policies, following best practices, developing and possessing an in-depth understanding of exploits and vulnerabilities, resolving issues by taking appropriate corrective action.
• Knowledge of security controls designing Source and Data Transfers including CRON, ETLs, and JDBC-ODBC scripts.
• Understand basics of Networking including DNS, Proxy, ACL, Policy and troubleshooting
• High level knowledge of compliance and regulatory requirements of data including but not limited to encryption, anonymization, data integrity, policy control features in large scale infrastructures
• Understand data sensitivity in terms of logging, events and in memory data storage– such as no card numbers or personally identifiable data in logs.
• Implement wrapper solutions for new/existing components with no/minimal security controls to ensure compliance to bank standards.