Data Engineer Level 2 | 3-5 YOE | Immediate Joiners | Bangalore

About the Role

Our client is seeking a Data Engineer to build high-performance pipelines that power their Explainable AI engine. In this role, you will be responsible for ingesting, normalizing, and modeling complex billing metadata from AWS, Azure, GCP, and Snowflake, ensuring that autonomous agents operate on a foundation of high-fidelity data. A FinOps background is not required, as the domain is learnable, and the team will provide the necessary ramp-up.

Key Responsibilities

• Ingest raw structured and semi-structured data from customer cloud environments (AWS, Azure, GCP) into Snowflake staging layers.
• Configure ADF Linked Services, Integration Runtimes, and pipeline triggers for reliable data movement across multi-cloud boundaries.
• Optimize pipeline performance for high-volume datasets, ensuring low-latency delivery to downstream transformation layers.
• Pull data from REST APIs and third-party data sources as required by product needs.
• Transform and normalize API responses using Python and SQL to produce clean, structured tables in Snowflake.
• Ensure transformed data is structured and optimized for direct consumption by front-end applications and BI tools.
• Build reusable Python utilities for API pagination, error handling, and incremental data extraction.
• Collaborate with product and domain experts to gather and analyze business requirements for new product modules.
• Translate business requirements into clean, scalable data models.
• Design analytics-ready schemas (Star Schema, dimensional models) for ML and front-end consumption.
• Implement data quality frameworks to ensure accuracy, completeness, and consistency across all pipeline layers.
• Propose refresh strategies for each table, identifying appropriate SCD types.
• Implement automated testing and watermarking logic to prevent data gaps in Financial Reporting.
• Instrument observability metadata for monitoring data quality and performance.

Required Skills

• 3–5 years in a Data Engineering role; SaaS or analytics product experience is a plus.
• Hands-on experience with Azure Data Factory (Linked Services, IR, triggers, parameterization).
• Proficiency in Snowflake, including virtual warehouses and role-based access.
• Expert-level SQL for complex transformations; strong Python for API integrations and data validation.
• Ability to design Star Schemas and handle SCDs (Type 1 & 2).
• Experience with REST APIs, including pagination and authentication.
• Knowledge of data quality checks and freshness monitoring.
• Familiarity with AWS, Azure, or GCP environments.

Preferred Skills

• Experience with Infrastructure as Code (IaC) tools like Terraform or CloudFormation.
• Knowledge of query optimization and indexing in distributed environments.
• Exposure to real-time data tools like Kafka, Kinesis, or Spark Streaming.
• Experience building or consuming RESTful APIs.
• Familiarity with Docker and Kubernetes from a data workload perspective.

Qualifications

• Strong analytical thinking and problem-solving skills.
• Detail-oriented with a focus on data integrity.
• Excellent communication skills and ability to empower team collaboration.
• Intellectual curiosity and a fast learning ability.

Why Join?

• Gain exposure to enterprise-scale cloud billing datasets.
• Become a subject matter expert in Multi-Cloud FinOps.
• Acquire hands-on experience in AI-native product building.
• Enjoy architectural ownership and influence the long-term roadmap in a high-growth startup.
• Develop business and financial acumen, bridging technical infrastructure with business ROI.

About YMinds.AI

YMinds.AI is a premier talent solutions company specializing in sourcing and delivering elite developers with expertise in cutting-edge technologies. We help global enterprises and fast-growing startups accelerate their product development by connecting them with engineers who excel in building intelligent, scalable, and future-ready systems. Our clients are at the forefront of innovation, and we enable their success by providing exceptional talent that drives technological advancement.

Keywords

Data Engineer, Azure Data Factory, Snowflake, SQL, Python, REST APIs, Cloud Billing, Multi-Cloud, Data Quality, Star Schema, FinOps, AI Economics, Data Modeling, Pipeline Development, #DataEngineering #CloudComputing #FinOps #AI #Snowflake #Azure #Python #SQL #JobDescription