Validation

To validate your model following techniques are available:

K-Fold Validation

It is running k-fold cross-validation. It accepts parameters:

• k_folds - the number of folds in cross-validation. The default is set to 5.
• stratify - decides if samples will be distributed equally in classes between training and validation subsets. It is available only for classification tasks. The default is set to False.
• shuffle - decides if shuffle samples before training, The default set to True.
• random_seed - a seed

To use this validation please set validation_type="kfold". In this validation scenario, there will be created k_folds of machine learning models (if you set k_folds=5 then as a result 5 models will be trained on different portions of data).

Example paramaters for k-fold validation

{
  "validation_type": "kfold",
  "k_folds": 5,
  "shuffle": True,
  "stratify": False,
  "random_seed": 123
}

Split Validation

It accepts parameters:

• train_ratio - what ratio of samples should be used for training. Should be in 0 - 1 range.
• stratify - decides if samples will be distributed equally in classes between training and validation subsets. It is available only for classification tasks. The default is set to False.
• shuffle - decides if shuffle samples before training, The default set to True.
• random_seed - a seed

To use this validation please set validation_type="split". In this validation scenario, there will be created one machine learning model.

Example paramaters for train/validation split

{
  "validation_type": "split",
  "train_ratio": 0.8,
  "shuffle": True,
  "stratify": False,
  "random_seed": 123
}

Validation with a separate dataset (validation dataset)

To use this validation please set validation_type="with_dataset". In this validation scenario, there will be created one machine learning model. Please make sure that you provide train and validation datasets. There should be different datasets, not the same!

Example parameters for validation with separate dataset

{
  "validation_type": "with_dataset",
}