Support Window: 6:00 PM-10:00 AM- Assignment: 3 Hours Daily- Mode: Remote / Part-Time- Focus: Databricks + Machine LearningROLE OVERVIEW - We are looking for an experienced Senior Databricks / Machine Learning Engineer with 8+ years of experience for a part-time job support assignment.- The ideal candidate must have strong hands-on experience in Azure, Azure Databricks, Apache Spark, PySpark and Machine Learning, with experience building data ingestion, processing, ML experimentation and deployment workflows.- Candidates must be available for a 3-hour assignment within the 6:00 PM- 10:00 AM support window.- MUST-HAVE TECHNOLOGIES - Cloud: Microsoft Azure Data Ingestion: Azure Synapse Pipelines Storage: Azure Data Lake Storage (ADLS) Data Processing: Azure Databricks Apache Spark PySpark Data Formats: Delta Lake / Parquet Machine Learning: Python Keras Machine Learning model development Hyperparameter Tuning: Optuna Experiment Tracking: MLflow Model Registry: MLflow Model Registry Workflow Orchestration: Databricks Jobs YAML Configurations Source Control: GitHub Artifact Management: JFrog Artifactory Documentation: Confluence Operational / AI Analysis: Genie Agents Future custom-agent ecosystem Deployment: Plant Servers Supporting Systems: DTR BOPH / Equipment Metadata Plant Telemetry SystemsKEY RESPONSIBILITIES - Develop and support Azure Databricks data and ML solutions.- Build scalable data processing pipelines using Spark/PySpark.- Work with Azure Synapse Pipelines for data ingestion.- Process and manage data in ADLS using Delta/Parquet formats.- Develop ML solutions using Python and Keras.- Implement hyperparameter optimization using Optuna.- Manage ML experiments and model lifecycle using MLflow.- Maintain MLflow Model Registry.- Configure and manage Databricks Jobs and YAML-based workflows.- Use GitHub for source control and JFrog Artifactory for artifact management.- Support deployment of models/solutions to plant servers.- Work with plant telemetry, equipment metadata, DTR and BOPH systems.- Support operational analysis involving Genie agents and future custom-agent ecosystems.- Troubleshoot data, ML and deployment issues.- Maintain technical documentation in Confluence.CANDIDATE MUST HAVE- 8+ years overall IT experience- Strong Azure Databricks experience- Strong Machine Learning experience- Solid Python + PySpark + Apache Spark- Azure Data Lake Storage experience- Azure Synapse Pipelines experience- Keras / ML model development experience- Optuna / hyperparameter tuning experience- MLflow + MLflow Model Registry experience- Databricks Jobs + YAML experience- GitHub experience- Experience with Delta / Parquet- Good understanding of ML lifecycle and deployment- Experience working with production/operational systems- Available for 3 hours daily- Available within 6 PM-10 AM support window.

Senior Databricks / Machine Learning Engineer (Hyderabad)

Job Description & Responsibilities

About Supreme Consulting

Required Skills

Job Details

Posted by

About Supreme Consulting

More open roles