Lead Data Engineer SQL AWS

Talworx is an emerging recruitment consulting and services firm , we are hiring for our Fintech Product based client, In this role, you will be working with a team or teams of enthusiastic members supporting our critical technology systems and guiding our business partners & end users with industry best practices, solution design, and creating long term value to our customers.

You will work in a (truly) global team and encouraged for thoughtful risk-taking and self-initiative.

As a member of the Cognitive Engineering team, you will build and maintain enterprise-scale data extraction, automation, and ML model deployment pipelines. You will design resilient, production-ready systems within an AWS-based ecosystem, collaborating with a hands-on, technically strong global team to solve high-complexity problems end-to-end.

Develop, deploy, and operate data extraction and automation pipelines in production.

Integrate and deploy machine learning models into pipelines (e.g., Lead the delivery of complex extraction, transformation, and ML deployment projects.

Scale pipelines on AWS (EKS, ECS, Lambda) and manage DataOps processes with Celery, Redis, and Airflow.

Implement robust CI/CD pipelines on Azure DevOps and maintain comprehensive test coverage.

Strengthen data quality and reliability through logging, metrics, and automated alerts.

Partner with data scientists, ML engineers, and product teams to align on requirements and delivery timelines.

2-6 years of relevant experience in data engineering, automation, or ML deployment.

• Expert proficiency in Python, including building extraction libraries and RESTful APIs.
• Hands-on experience with task queues and orchestration: Strong AWS expertise: Containerization and orchestration experience: Docker (mandatory), basic Kubernetes (preferred).
• Proven experience deploying ML models to production.
• Solid understanding of CI/CD practices and hands-on experience with Azure DevOps.
• Familiarity with SQL and NoSQL stores (e.g., PostgreSQL, MongoDB)