-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add array or regex to data #23
Comments
Hm, good question. I like the At the moment I'm also in favor of allowing wildcards then, as this allows to be stricter in defining the file names (e.g. So I like the flexibility of the wildcard together with the clarity of the |
I also like the flag idea. We could test this on the catflow generator tool where I think multiple tiff files have to be read in as input for the tool. |
In the data section, there might be a case, in which not only a single file is associated to an input dataset, but a list of files.
In these cases we can either:
array
similar to the parametersIn both cases, the specs cannot handle cases in which the number of datasets is arbitrary. In these cases, the developer has to fall back to specify a directory in the parameters instead of data section.
An example for the multi-files case: A tool takes a netCDF, which is chunked into many files
An example for the multi-dataset (multi-files): An aggregator or viewer tool takes a folder as input, that contains data folders. Similar to what the data loader creates.
I would argue, that this is an edge case and usually tools can specify the data they need.
I am in preference of setting ie. a
multi=True
flag on a data spec, which effectively allows wildcards in the path@Ash-Manoj @AlexDo1 do you have any comments on this? I am not entirely sure how to do that and comments are welcome
The text was updated successfully, but these errors were encountered: