📅 2024-04-15 — Session: Integrated MLflow and DVC for MLOps Enhancement
🕒 15:45–17:20
🏷️ Labels: Mlflow, DVC, Mlops, Flask, Docker
📂 Project: Dev
⭐ Priority: MEDIUM
Session Goal
The session aimed to enhance machine learning operations (MLOps) by integrating MLflow and DVC for better model and data versioning, experiment tracking, and deployment.
Key Activities
- Explored various MLOps tools like MLflow, DVC, and Kubeflow, focusing on their capabilities in model and data versioning.
- Created a workflow diagram using Graphviz to visualize the integration of MLflow and DVC.
- Configured MLflow tracking URI and set up MLflow in Docker for experiment tracking.
- Enhanced a Flask endpoint with MLflow for improved model retraining and experiment tracking.
- Resolved MLflow-related issues such as
PermissionError, tracking URI path issues, and experiment not found errors. - Implemented a naming convention for MLflow experiments to ensure meaningful and unique experiment identifiers.
- Addressed file path discrepancies in Python for model saving and loading, and improved model loading in Flask applications.
- Updated model retrieval and dropdown selection in the application.
Achievements
- Successfully integrated MLflow and DVC into the workflow, enhancing model and data management.
- Resolved several configuration and error handling issues related to MLflow and Flask integration.
- Established a robust framework for experiment tracking and model management in MLOps.
Pending Tasks
- Further testing of the integrated system to ensure stability and performance.
- Documentation of the setup and integration process for future reference.