logger.md

ClearML Explicit Logging

Using the ClearML Logger module and other ClearML features, you can explicitly log any of the following:

• Report graphs and images

  • Scalar metrics
  • Histograms
  • Line plots
  • 2D scatter diagrams
  • 3D scatter diagrams
  • Confusion matrices
  • Surface diagrams
  • Images

• Track hyperparameters and OS environment variables

  • Logging experiment parameter dictionaries
  • Specifying environment variables to track

• Message logging

  • Reporting text without formatting

Additionally, the ClearML Logger module provides methods that allow you to do the following:

• Get the current logger
• Override the ClearML configuration file with a default upload destination for images and files

Graphs and Images

Scalar Metrics

Report scalar metrics by iteration as a line plot.

First get the current logger and then use it (see an example script) with the following method.

Method:

def report_scalar(self, title, series, value, iteration)

Arguments:

Parameter	Type	Description	Mandatory
iteration	integer	The iteration number (x-axis).	Yes
series	string	The data series label.	Yes
title	string	The plot title.	Yes
value	float	The scalar metric data value (y-axis).	Yes

Histograms

Report any data by iteration as a histogram.

First get the current logger and then use it (see an example script) with the following method.

Method:

def report_histogram(self, title, series, values, iteration, labels=None, xlabels=None)

Arguments:

Parameter	Type	Description	Mandatory
iteration	integer	The iteration number (x-axis).	Yes
series	string	The data series label.	Yes
title	string	The plot title.	Yes
values	Either: list of float numpy array	The histogram data values (y-axis).	Yes
labels	list of strings	Labels for each bar group in the histogram. The default value is None.	No
xlabels	list of strings	Labels for each bucket in the histogram. Each label in the xlabels list corresponds to a value in the values list (or numpy array). The default value is None.	No

Line Plots

Report any data by iteration as a single or multiple line plot.

First get the current logger and then use it (see an example script) with the following method.

Method:

def report_line_plot(self, title, series, iteration, xaxis, yaxis, mode='lines', reverse_xaxis=False, comment=None)

Arguments:

Parameter	Type	Description	Mandatory
iteration	integer	The iteration number (x-axis).	Yes
series	LineSeriesInfo	One (single line plot) or more (multiple line plot) series of data.	Yes
title	string	The plot title.	Yes
comment	string	Any text (e.g., subtitle or other comment) which displays under the plot title. The default value is None.	No
mode	string	Type of line plot. The values are: lines - lines connecting data points (default) markers - markers for each data point lines+markers - lines and markers	No
reverse_xaxis	boolean	Indicates whether to display the x-axis values in ascending or descending order. The values are: True - descending order False - ascending order (default)	No
xaxis	string	x-axis title.	No
yaxis	string	y-axis title.	No

2D Scatter Diagrams

Report any vector data as a 2D scatter diagram.

First get the current logger and then use it (see an example script) with the following method.

Method:

def report_scatter2d(self, title, series, scatter, iteration, xaxis=None, yaxis=None, labels=None, mode='lines', comment=None)

Arguments:

Parameter	Type	Description	Mandatory
iteration	integer	The iteration number (x-axis).	Yes
scatter	Either: list of (pairs of x, y) ndarray	The scatter data. For example, a list of pairs in the form: [(x1,y1),(x2,y2),(x3,y3),...].	Yes
series	string	The data series label.	Yes
title	string	The plot title.	Yes
comment	string	Any text (e.g., subtitle or other comment) which displays under the plot title. The default value is None.	No
labels	list of strings	Label text per data point in the scatter diagram. The default value is None.	No
mode	string	Type of 2D scatter diagram. The values are: lines - lines connecting data points (default) markers - markers for each data point lines+markers - lines and markers	No
xaxis	string	x-axis title. The default value is None.	No
yaxis	string	y-axis title. The default value is None.	No

3D Scatter Diagrams

Report any array data as a 3D scatter diagram.

First get the current logger and then use it (see an example script) with the following method.

Method:

def report_scatter3d(self, title, series, scatter, iteration, labels=None, mode='markers', fill=False, comment=None)

Argument:

Parameter	Type	Description	Mandatory
iteration	integer	The iteration number (x-axis).	Yes
scatter	Either: list of (pairs of x, y, z) ndarray list of series [[(x1,y1,z1)...]]	The scatter data. For example, a list of series in the form: [[(x1,y1,z1),(x2,y2,z2),...],[(x3,y3,z3),(x4,y4,z4),...],[(x5,y5,z5),(x6,y6,z6),...]]	Yes
series	string	The data series label.	Yes
title	string	The plot title.	Yes
comment	string	Any text (e.g., subtitle or other comment) which displays under the plot title. The default value is None.	No
fill	boolean	Indicates whether to fill the area under the scatter diagram. The values are: True - fill False - do not fill (default)	No
labels	list of strings	Label text per data point in the scatter. The default value is None.	No
mode	string	Type of 3D scatter diagram. The values are: lines markers lines+markers The default value is lines.	No

Confusion Matrices

Report a heat-map matrix as a confusion matrix. You can also plot a heat-map as a surface diagram.

First get the current logger and then use it (see an example script) with the following method.

Method:

def report_confusion_matrix(self, title, series, matrix, iteration, xlabels=None, ylabels=None, comment=None)

Arguments:

Parameter	Type	Description	Mandatory
iteration	integer	The iteration number (x-axis).	Yes
matrix	ndarray	A heat-map matrix.	Yes
series	string	The data series label.	Yes
title	string	The plot title.	Yes
xlabels	list of strings	Label per column of the matrix. The default value is None.	No
ylabels	list of strings	Label per row of the matrix. The default value is None.	No

Surface Diagrams

Plot a heat-map matrix as a surface diagram. You can also plot a heat-map as a confusion matrix.

First get the current logger and then use it (see an example script) with the following method.

Method:

def report_surface(self, title, series, matrix, iteration, xlabels=None, ylabels=None, xaxis=None, yaxis=None, camera=None, comment=None)

Arguments:

Parameter	Type	Description	Mandatory
iteration	integer	The iteration number (x-axis).	Yes
matrix	ndarray	A heat-map matrix.	Yes
series	string	The data series label.	Yes
title	string	The plot title.	Yes
camera	tuple	The position of the camera as (x, y, x), if applicable.	No
comment	string	Any text (e.g., subtitle or other comment) which displays under the plot title. The default value is None.	No
xlabels	list of strings	Label per column of the matrix. The default value is None.	No
xaxis	string	x-axis title.	No
ylabels	list of strings	Label per row of the matrix. The default value is None.	No
yaxis	string	y-axis title.	No

Images

Report an image and upload its contents to the bucket specified in the ClearML configuration file, or a default upload destination, if you set a default.

First get the current logger and then use it (see an example script) with the following method.

Method:

def report_image(self, title, series, iteration, local_path=None, matrix=None, max_image_history=None, delete_after_upload=False)

Arguments:

Parameter	Type	Description	Mandatory
iteration	integer	The iteration number.	Yes
series	string	The label of the series.	Yes
title	string	The title of the image.	Yes
delete_after_upload	boolean	Indicates whether to delete the local copy of the file after uploading it. The values are: True - delete False - do not delete (default)	No
matrix	ndarray	A 3D numpy.ndarray object containing image data (RGB). If path is not specified, then matrix is required. The default value is None.	No
max_image_history	integer	The maximum number of images to store per metric / variant combination. For an unlimited number of images to store, specify a negative number. The default value which is set in the global configuration is 5.	No
local_path	string	The path of the image file. If matrix is not specified, then path is required. The default value is None.	No

Hyperparameters and Environment Variables

Logging Experiment Parameter Dictionaries

In order for ClearML to log a dictionary of parameters, use the Task.connect method.

For example, to log the hyperparameters learning_rate, batch_size, display_step, model_path, n_hidden_1, and n_hidden_2:

# Create a dictionary of parameters
parameters_dict = { 'learning_rate': 0.001, 'batch_size': 100, 'display_step': 1, 
    'model_path': "/tmp/model.ckpt", 'n_hidden_1': 256, 'n_hidden_2': 256 }
    
# Connect the dictionary to your ClearML Task    
parameters_dict = Task.current_task().connect(parameters_dict)

Specifying Environment Variables to Track

By setting the CLEARML_LOG_ENVIRONMENT environment variable, make ClearML log either:

• All environment variables

    export CLEARML_LOG_ENVIRONMENT=*
  

• Specific environment variables

  For example, log PWD and PYTHONPATH

    export CLEARML_LOG_ENVIRONMENT=PWD,PYTHONPATH
  

• No environment variables

    export CLEARML_LOG_ENVIRONMENT=
  

Logging Messages

Use the methods in this section to log various types of messages. The method name describes the type of message.

Debugging Messages

Method:

def debug(self, msg, *args, **kwargs)

First get the current logger and then use it (see an example script) with the following method.

Arguments:

Parameter	Type	Description	Mandatory
msg	string	The text to log.	Yes

Informational Messages

Method:

def info(self, msg, *args, **kwargs)

First get the current logger and then use it (see an example script) with the following method.

Arguments:

Parameter	Type	Description	Mandatory
msg	string	The text to log.	Yes

Warnings

Method:

def warn(self, msg, *args, **kwargs)

First get the current logger and then use it (see an example script) with the following method.

Arguments:

Parameter	Type	Description	Mandatory
msg	string	The text to log.	Yes

General Errors

Method:

def error(self, msg, *args, **kwargs)

First get the current logger and then use it (see an example script) with the following method.

Arguments:

Parameter	Type	Description	Mandatory
msg	string	The text to log.	Yes

Critical Errors

Method:

def critical(self, msg, *args, **kwargs)

First get the current logger and then use it (see an example script) with the following method.

Arguments:

Parameter	Type	Description	Mandatory
msg	string	The text to log.	Yes

Fatal Errors

Method:

def fatal(self, msg, *args, **kwargs)

First get the current logger and then use it (see an example script) with the following method.

Arguments:

Parameter	Type	Description	Mandatory
msg	string	The text to log.	Yes

Console and Logger Messages

Method:

def console(self, msg, level=logging.INFO, omit_console=False, *args, **kwargs)

First get the current logger and then use it (see an example script) with the following method.

Arguments:

Parameter	Type	Description	Mandatory
msg	string	The text to log.	Yes
level	string	The log level. The values are: logging.DEBUG logging.INFO logging.WARNING logging.ERROR logging.FATAL logging.CRITICAL	No
omit_console	boolean	Indicates whether to send the message to the log only. The values are: True - send the message to the log only False - send the message to the console and the log	No

Reporting Text Without Formatting

Method:

def report_text(self, msg, level=logging.INFO, print_console=False, *args, **_)

First get the current logger and then use it (see an example script) with the following method.

Arguments:

Parameter	Type	Description	Mandatory
msg	string	The text to log.	Yes
level	string	The log level. The values are: logging.DEBUG logging.INFO logging.WARNING logging.ERROR logging.FATAL logging.CRITICAL	No
print_console	boolean	Indicates whether to log to the console, in addition to the log. The values are: True - print to the console and log False - print to the log, only (default)	No

Logger Object and Storage Methods

Get the Current Logger

Use to return a reference to the current logger object.

Method:

def current_logger(cls)

Arguments:

None.

Set Default Upload Destination

Specify the default destination storage location used for uploading images. Images are uploaded and a link to the image is reported.

Credentials for the storage location are in the global configuration file (for example, on Linux, ~/clearml.conf).

Method:

def set_default_upload_destination(self, uri)

Arguments:

Parameter	Type	Description	Mandatory
uri	string	The destination storage location, for example s3://bucket/directory/ or file:///tmp/debug/.	Yes