Data Engineer Lead (Python, AWS, SQL)

Years of Experience - 8-12 years

Location - Trivandrum

Job description

We are looking for Senior ETL / Data Integration Engineers to reinforce an existing 3- person ETL team. The team is responsible for designing, developing and maintaining all data flows between the E-commerce platform, SAP, the OMS and several third-party systems. The technical environment is mainly based on Python, AWS Lambda, AWS ECS and eventdriven pipelines triggered through a Kafka-like messaging system. The objective is to onboard senior, hands-on engineers who can quickly contribute to existing developments, improve integration patterns and bring technical expertise to support the team’s roadmap.

Responsibilities

• Design, develop and maintain ETL pipelines and data integration flows
• Build and improve Python-based data processing components
• Develop and operate AWS Lambda functions and containerized services on ECS
• Integrate data flows between E-commerce, SAP, OMS and external systems
• Work with event-driven mechanisms to trigger and orchestrate ETL pipelines
• Ensure reliability, scalability, data consistency and proper error handling
• Contribute to CI/CD pipelines and delivery processes using GitLab/Terraform
• Participate in code reviews, technical discussions and architecture decisions
• Improve observability, logging and monitoring of ETL flows
• Document technical solutions and integration patterns

Required Experience

• Senior hands-on engineer with strong Python development experience
• Solid experience in AWS cloud-native environments
• Good knowledge of Lambda, ECS and containerized workloads
• Experience with ETL, data integration or event-driven architectures
• Understanding of messaging systems such as Kafka or equivalent
• Ability to analyze existing systems and propose pragmatic improvements
• Strong autonomy, ownership and delivery mindset
• Good communication skills with technical and non-technical stakeholders
• Comfortable working in an agile environment with GitLab, Jira and Confluence

Nice to Have

• Experience in E-commerce environments
• Experience with SAP or OMS integration
• Knowledge of data quality, retry mechanisms, idempotency and monitoring best practices

Skills

ETL, Python, Amazon ECS, AWS Lambda