Databricks Data Engineer (WFH 100%)

Job Introduction

We are seeking a skilled Azure Databricks Data Engineer to join our data engineering team. The ideal candidate will have hands-on experience designing and maintaining scalable data pipelines using Azure Databricks and Data Factory, strong technical skills in Python and SQL, and familiarity with machine learning concepts. This role requires a proactive, solution-oriented professional who can collaborate effectively with multicultural teams and contribute to high-quality data solutions.

Job Description

As an Azure Databricks Data Engineer, your responsibilities include

• Technical Requirements Gathering and Development of Functional Specifications.
• Designing, developing, and maintaining scalable data pipelines and ETL processes using Azure Databricks, Data Factory, and other Azure services.
• Implementing and optimizing Spark jobs, data transformations, and data processing workflows in Databricks.
• Developing and integrating custom machine learning models using Azure Machine Learning, MLflow, and other relevant libraries.
• Leveraging Azure DevOps and CI/CD best practices to automate the deployment and management of data pipelines and infrastructure.
• Conducting troubleshooting on data models.
• Working with Agile multicultural teams in Asia, the EU, Canada, and the USA.

Profile Requirements For this position of Azure Databricks Data Engineer, we are looking for someone with:

• (Required) At least 5 years of experience in developing and maintaining data pipelines using Azure Databricks, Azure Data Factory, and Spark.
• (Required) Hands-on experience with Unity Catalog.
• (Required) Fluent English communication and soft skills.
• (Required) Knowledge and experience in CI/CD using tools such as Terraform, ARM, and Bicep Script.
• (Required) Solid technical skills in Python and SQL.
• (Required) Familiarity with machine learning concepts, tools, and libraries (e.g., TensorFlow, PyTorch, Scikit-learn, MLflow).
• (Required) Strong problem-solving, communication, and analytical skills.
• (Optional) Willingness to learn and expand technical skills in other areas.