Lead II - Data Engineering

• 5 to 8 years of Senior Software engineering experience, ideally in infrastructure or data platform roles.
• Proficiency in Python/PySpark and strong knowledge of SQL.
• Experience with Terraform or similar Infrastructure-as-Code (IaC) tools.
• Strong knowledge of at least one cloud platform (GCP/AWS)
• Solid CI/CD experience
• Data & MLOps Expertise:
• Strong Background in Databricks, specifically Unity Catalog, medallion architecture, and Databricks Asset Bundles (DABs).
• Proficiency in Python and API frameworks such as FastAPI or Flask.
• Strong experience designing, developing, and maintaining RESTful APIs.
• Good understanding of data engineering concepts such as data modeling and workflow orchestration.
• Experience with MLOps and Data Engineering like having worked on Spark/Delta Lake/MLflow/ETL frameworks (e.g., Fivetran, Lakeflow Connect).
• Operational Excellence:
• Experience with Github workflows, including container versioning.
• Proficiency with incident management and observability tools (e.g., PagerDuty, system monitoring).
• An AI as a partner mindset.
• A collaborative approach, including asking questions, navigating unclear requirements, and actively contributing to knowledge sharing/community enablement.

Role Description

• 5 to 8 years of Senior Software engineering experience, ideally in infrastructure or data platform roles.
• Proficiency in Python/PySpark and strong knowledge of SQL.
• Experience with Terraform or similar Infrastructure-as-Code (IaC) tools.
• Strong knowledge of at least one cloud platform (GCP/AWS)
• Solid CI/CD experience
• Data & MLOps Expertise:
• Strong Background in Databricks, specifically Unity Catalog, medallion architecture, and Databricks Asset Bundles (DABs).
• Proficiency in Python and API frameworks such as FastAPI or Flask.
• Strong experience designing, developing, and maintaining RESTful APIs.
• Good understanding of data engineering concepts such as data modeling and workflow orchestration.
• Experience with MLOps and Data Engineering like having worked on Spark/Delta Lake/MLflow/ETL frameworks (e.g., Fivetran, Lakeflow Connect).
• Operational Excellence:
• Experience with Github workflows, including container versioning.
• Proficiency with incident management and observability tools (e.g., PagerDuty, system monitoring).
• An AI as a partner mindset.
• A collaborative approach, including asking questions, navigating unclear requirements, and actively contributing to knowledge sharing/community enablement.

Skills

• AWS
• Databricks
• PySpark
• Python
• CI/CD
• FastAPI
• Docker
• Infrastructure as Code
• Terraform
• PagerDuty
• ETL