GUIDE.md

Experiments

Experiments are, like projects, defined in yaml configuration files, we denote such files; experiment configuration files (ECFs). To avoid having multiple similar ECFs (and thus avoiding cluttering your project) Tracker will store all attributes given for each trial conducted in the Tracker home folder. This means that you can do a lot of testing without having to worry about which file gave which result, these are automatically linked together in the Tracker home folder. When you want to share a specific trial with a colleague you can simply export it into a yaml file and send that.

You can create your ECFs in one of three ways; (1) directly from a trial, (2) manually or (3) automatic extraction of parameters from Python files:

1. Direct creation of an ECF from starting a trial

tracker experiment run EXECUTABLE_FILE:OPERATION_NAME parameters.seed.value=42 parameters.batch_size.value=42 [...]

2. Manually (before run)

Fire up you favorite editor and start creating your ECFs.

3. Automatically through AST (iif code is Python)

tracker experiment create EXPERIMENT_NAME:OPERATION_NAME -f PYTHON_EXE_FILE

Example ECF

- experiment: vgg16
  description: Using Tracker to do good science!
  operations:
    train:
      main: src/train.py
      seed: 42            # [optional]
      parameters:         # [optional]
        batch_size:
          description: Number of images pr. batch
          value: 32
        lr:
          description: Learning rate
          min: 1e-4
          max: 0.1
        [...]
      test:
        main: ...
        description: ...
        [...]

Experiments are linked from the project configuration file tracker.yaml located at the root of the project (auto-generated by Tracker).

experiments:
- vvg16:
    path: experiments/vgg16.yaml
- vgg32:
    path: experiments/vgg32.yaml
[...]

Remotes

Remotes are defined either in the global Tracker configuration file (~/.tracker/config.yaml) or for project specific remotes; in the root of the project directory ([...]/tracker.yaml).

Each remote is named under the remotes section key.

remotes:
  remote-1:
    ...
  remote-2:
    ...
  remote-n:
    ...

General Attributes

Attributes	Description
description	Description (string) of the server

SSH Attributes

Attributes	Description
host	Server host name or IP address (required string)
port	Server SSH port (number) By default, port 22 is used for SSH connections.
user	User used when connecting over SSH (string) By default, the active user name is used for SSH connections.
private-key*
tracker‑home	Path to Tracker home on the remote server (string).

To add your private key to the SSH server add your public key to the list of authorised keys by:

cat ~/.ssh/id_rsa.pub | ssh user@192.168.0.1 'cat >> .ssh/authorized_keys'