ClearML with Nvidia Frameworks

How to Use ClearML with Nvidia's TLT, Clara and rapidsai quickly and easily - no setup required!

Introduction

This repository provides Deep Learning examples showing how use ClearML to easily run Nvidia's Clara, TLT and Rapidsai frameworks. It includes an example for each Nvidia framework,

Each example has a ready-made experiments in the Free ClearML Hosted Service as well as in the ClearML Demo Server, and can be quickly cloned, configured and executed using the ClearML WebApp.

Run the examples

Set up a ClearML account

To run these examples, you need to use a ClearML Server containing the ready-made example experiments. For example, you can open an account (or use an existing account) in the Free ClearML Hosted Service. See Getting Started using the Free ClearML Hosted Service for more information

Install ClearML Agent

In order to run the experiments, you'll need ClearML Agent installed on a machine with an Nvidia GPU. For installation instructions, see Installing and Configuring Your ClearML Agent.

Create a Dataset (optional)

If you'd like to use your own data, you can create a new dataset and edit it using the clearml-data command-line interface.

Generally, the clearml-data flow is Create -> Add -> Upload -> Close -> Publish (optional).

To create your own dataset:

1. Install the clearml package (this also installs the clearml-data command):

   pip install clearml

2. Configure ClearML (make sure to obtain your credentials from the account you previously set up):

   clearml-init

3. Create a new dataset:

   clearml-data create --project "TLT with ClearML" --name "Example data"

4. Add files to your new dataset:

   clearml-data add --files /home/datasets/my_dataset_for_tlt.zip

5. Upload the files:

   clearml-data upload

6. Close the dataset task (from this point onward, the dataset will be read-only):

   clearml-data close

Note: for more information on the various command line options, see clearml-data --help

Prepare the experiment

In order to run an example, you should use the ClearML WebApp to:

1. Clone the base experiment for that example
2. Modify the parameters as you see fit
3. Enqueue the experiment
4. The ClearML-agent listening to your queue will run the experiment, no code or any environment setup is required!

What will I see when running the examples?

For each example, the ClearML WebApp will show:

• Full console output
• Any reported Scalars
• Artifacts (Models, result tables and more)
• Experiment arguments
• Experiment configuration

Run without the pre-made example experiments

See each of the frameworks READMEs for instructions on how to run each example. Please note that you will need to run each experiment using the appropriate docker image for the framework in question.

Documentation, Community & Support

More information in the official documentation and on YouTube.

For examples and use cases, check the examples folder and corresponding documentation.

If you have any questions: post on our Slack Channel, or tag your questions on stackoverflow with 'clearml' tag (previously trains tag).

For feature requests or bug reports, please use GitHub issues.

Additionally, you can always find us at clearml@allegro.ai