list of a default config

[de-gozaru]

Hi,
I'm using Hydra 1.1 for deep learning application.
My conf directory has the following architecture:

├── conf
│   ├── config.yaml
│   ├── dataset
│   │   ├── imagenet.yaml
│   │   └── pascal.yaml
│   ├── architecture
│   │   ├── resnet.yaml
│   │   ├── alexnet.yaml
│   └── optimizer
│          ├── adam.yaml
│          └── sgd.yaml
└── main.py

my config.yaml has the following structure:

defaults:
  - dataset@trainset: imagenet
  - dataset@testset: imagenet
  - architecture: resnet
  - optimizer: adam

What I want to do is that, sometimes I want to use multiple train dataset, something like this

python3 main.py dataset@trainset=imagenet trainset.remove_class=cat

OR

python3 main.py dataset@trainset=[imagenet, pascal] trainset.remove_class=cat

is this possible with the current version? if not, is there a possible turnaround I can use?
I know that an append function is scheduled to be added in version 1.3.

Thank you in advance for your help!

[jieru-hu]

hi @de-gozaru
instead of using a list, try using a dict instead, it is better supported in Hydra.

would something like the following works?

python main.py +dataset@trainset.one=imagenet +dataset@trainset.two=pascal

then in your application, you can convert trainset's value into a list if needed.

[Jasha10]

Another option is to pass a list of config names at the command line:

$ python main.py 'dataset@trainset=[imagenet, pascal]'
trainset:
  imagenet:
    foo: bar
  pascal:
    baz: quz

Passing a list [imagenet, pascal] will cause the imagenet.yaml and pascal.yaml input configs to be merged; the result of the merge is placed in the trainset package. Note the single quotes in the above shell command may be necessary (depending on your shell). This example is working with the following setup:

from omegaconf import DictConfig, OmegaConf
import hydra

@hydra.main(config_path="conf", config_name="config")
def my_app(cfg: DictConfig) -> None:
    print(OmegaConf.to_yaml(cfg))

if __name__ == "__main__":
    my_app()

# conf/config.yaml
defaults:
  - dataset@trainset: ???

trainset: ???

# conf/dataset/imagenet.yaml
imagenet:
  foo: bar

# conf/dataset/pascal.yaml
pascal:
  baz: quz

As an alternative to passing a list of config names at the command-line (as in the example above), you could pass a list of config names in the config.yaml defaults list:

# conf/config.yaml
defaults:
  - dataset@trainset: [imagenet, pascal]

This is documented in the Defaults List page of the Hydra docs (see the note about "CONFIG_NAMES").

Edit: there is further documentation on passing a list of config names in Selecting multiple configs from a Config Group

[de-gozaru]

Hi @jieru-hu @Jasha10
Thanks a lot for your help, both solutions are ok for me!
I'll try them out and get back to you if I have any question.
closing the issue!