How to use mlflow with ultralytics ?

[Ashingharoy1991]

I've been exploring the Ultralytics documentation and saw that MLflow integration is possible by adjusting some settings. However, I’m encountering an issue when running training scripts in Python.

When I specify project and name arguments during training, the mlruns directory doesn't get created. But, if I skip these parameters, MLflow automatically creates the mlruns directory in the runs folder. Why does this happen? Is there a way to get consistent behavior with project and name specified?

Additionally, I’d like to change the default path for MLflow’s mlruns directory to a custom location. Is there a way to configure this through Ultralytics directly? Or I have to use mlflow?

Any insights would be appreciated!

[UltralyticsAssistant]

👋 Hello @Ashingharoy1991, thank you for reaching out with your questions about integrating MLflow with Ultralytics 🚀! We're here to help guide you in the right direction.

We recommend checking out our Docs for an overview of features and configurations. Specifically, for logging and integration details, you might find the Python usage examples helpful.

Since you're encountering an issue that could be a bug, it would be great if you could provide a minimum reproducible example of your script, including the configuration that leads to the behavior you’ve described. This will help us assist you more effectively.

For custom training setups and configuration questions, providing more details such as dataset specifics, your training commands, and any relevant logs can be incredibly useful. Make sure you're also following our Tips for Best Training Results.

Feel free to join our community in real-time on Discord 🎧 or share your experiences and get insights on our Discourse. Our Subreddit is also a great place for deeper dives into community knowledge sharing.

Upgrade

Make sure you're using the latest version of the ultralytics package to ensure all enhancements and bug fixes are included. Install or upgrade with:

pip install -U ultralytics

Supported Environments

YOLO works in several verified environments. Check out:

• Notebooks with free GPU:
• Google Cloud: Explore our GCP Quickstart Guide
• Amazon AWS: Review the AWS Quickstart Guide
• Docker Image: See our Docker Quickstart Guide

Status

The status badge indicates if all Ultralytics CI tests are passing, confirming the integrity of various Modes and Tasks across different operating systems.

This is an automated response, and an Ultralytics engineer will join the conversation soon to provide further assistance. 🙌

[Ashingharoy1991]

https://docs.ultralytics.com/integrations/mlflow/#introduction
In this section it's mentioned,how can I use mlflow

[glenn-jocher]

@Ashingharoy1991 to use MLflow with Ultralytics YOLO, ensure MLflow is installed, enable it in the settings, and start a local MLflow server for tracking.

[Ashingharoy1991]

@glenn-jocher hello!
I'm facing a problem with organizing the output from my YOLO training pipeline. I’ve set the project name and name parameters as suggested, expecting YOLO to create a structured directory with an mlrun folder and organize the training results under the specified project and run names. However, the following issues occur:

1. After training, the mlrun directory is not created within the specified project folder.

2. The results are not being reflected on the server as expected

[glenn-jocher]

Hello! It seems like there might be a configuration issue. Please ensure that MLflow logging is enabled in your settings and that the MLFLOW_TRACKING_URI is correctly set to point to your server. If the problem persists, try updating your Ultralytics package to the latest version.

[Ashingharoy1991]

@glenn-jocher hi, actually this is the step i am applying to used mlflow.. actually while i am doing training without specified project and name arguments , on that time in run folder the mlrun directory created and it store all information about run...

also i am using this code. but don't know how my pc get stuck in between the training

from ultralytics import YOLO
import mlflow
import os

Define the working directory for storing model artifacts

working_dir = "/home/st1/Mlops" # Define your working directory here

# Define the experiment ID for MLflow (make sure 'exp' is defined or replace it with an integer)

exp = 0 # Replace this with your actual experiment ID

Start an MLflow run

with mlflow.start_run(experiment_id=501178566973958999):
# Log data source information
mlflow.log_param("data_source", "MinIO")

# Define the YOLO model and start training
model = YOLO("yolov8n.pt")

# Set the path to the `data.yaml` file
data_yaml_path = "/home/st1/Mlops/yolo_v8.yaml"  # Replace with actual path if not dynamically generated
results = model.train(data=data_yaml_path, epochs=2, batch=16)

# Retrieve the metrics dictionary from the model
metrics = model.metrics.results_dict  # Access the dictionary of metrics

# Print the metrics dictionary for debugging
print("Metrics:", metrics)

# Log the relevant metrics to MLflow if they exist
precision = metrics.get('metrics/precision(B)', None)
if precision is not None:
    mlflow.log_metric("precision", precision)

recall = metrics.get('metrics/recall(B)', None)
if recall is not None:
    mlflow.log_metric("recall", recall)

mAP50 = metrics.get('metrics/mAP50(B)', None)
if mAP50 is not None:
    mlflow.log_metric("mAP50", mAP50)

mAP50_95 = metrics.get('metrics/mAP50-95(B)', None)
if mAP50_95 is not None:
    mlflow.log_metric("mAP50-95", mAP50_95)

fitness = metrics.get('fitness', None)
if fitness is not None:
    mlflow.log_metric("fitness", fitness)

# Log train and validation loss if available
train_loss = metrics.get("train_loss", None)
if train_loss is not None:
    mlflow.log_metric("train_loss", train_loss)

val_loss = metrics.get("val_loss", None)
if val_loss is not None:
    mlflow.log_metric("val_loss", val_loss)

# Save and log model weights
model_path = os.path.join(working_dir, "yolov8_model_weights.pt")  # Use working_dir for storing model
model.save(model_path)

# Log the saved model as an artifact in MLflow
mlflow.log_artifact(model_path, artifact_path="models")

[glenn-jocher]

@Ashingharoy1991 to ensure consistent behavior with MLflow when specifying project and name, make sure these parameters are correctly set in your environment or script. To change the default path for MLflow's mlruns directory, set the MLFLOW_TRACKING_URI environment variable to your desired location before starting the MLflow server. For more details, refer to the MLflow documentation.

[Ashingharoy1991]

@glenn-jocher hello!

here you can see i am changing the runs_dir patha but its not changing , can help me with it?

[glenn-jocher]

@Ashingharoy1991 hello! Yes, you should set the MLFLOW_TRACKING_URI environment variable before starting your training to specify a custom directory for MLflow logs. If it's still creating logs in the default directory, please ensure the variable is correctly set in your environment.

[Ashingharoy1991]

@glenn-jocher hello! the previous issue solved, thank you for your help,
another thing i want to know.

1. while i am doing training with this code
   
   the mlflow directory is created where i want to save the directory. now i have realised that the project arguments in yolo training actually the experiment name for mlflow and based on this project runs are store in that experiments.. what i am facing problem that lets say set directory is "/home/sti/test_mlflo" in this path the mlflow directory created and i want to save the project name folder also here so i run the in this directory so in this path now i have two directory one is "mlflow_test" and one is "mlflow". in the "mlflow" directory all the metadata are store for mlflow and in "mlflow_test" directory i have the yolo train weight folder and corresponding matrix images.. now if i open the mlflow server with this command "mlflow server --backend-store-uri /home/st1/test_mlflow/mlflow" then i can see all runs for the corresponding project and artifacts also , now the problem is that while i want to do model registry with the server UI , i am not getting any registry options , also i have tryout this command
   "mlflow server --backend-store-uri /home/st1/Mlops/test_project/ --default-artifact-root /home/st1/Mlops/test_project/838328226548121284/0c976120808246a48922057e5f5b33fc/artifacts"
   
   in this screen shots you can see there is no registry options i have
   

you can ignore the experiments name , problem is same

[glenn-jocher]

To enable model registry in MLflow, you'll need to use a proper backend database (like SQLite, PostgreSQL, or MySQL) instead of just a file system URI. Try starting the MLflow server with SQLite like this:

mlflow server --backend-store-uri sqlite:///mlflow.db --default-artifact-root /home/st1/test_mlflow/mlflow

This will create a SQLite database file and enable the model registry feature in the MLflow UI. The registry tab should then appear in the navigation menu.

[Ashingharoy1991]

@glenn-jocher hello.. thank you for your reply. actually i have already try this code. server is running but my previous experiments which is store in that path that's not showing .. its only showing while i am using local path to starting the server

[glenn-jocher]

To see your previous experiments in MLflow, make sure the --backend-store-uri points to the exact directory containing your MLflow tracking database (mlruns). The directory structure should be maintained between runs. You can verify the correct path by checking where your mlruns directory and metadata files are located.