Skip to content
This repository has been archived by the owner on Jun 3, 2024. It is now read-only.

Result folder of subpipelines. #15

Open
benHeid opened this issue Jun 22, 2021 · 4 comments
Open

Result folder of subpipelines. #15

benHeid opened this issue Jun 22, 2021 · 4 comments
Labels
bug Something isn't working

Comments

@benHeid
Copy link
Collaborator

benHeid commented Jun 22, 2021

Describe the bug
Each subpipeline stores its result in its own result folder.

To Reproduce
See Day and night example

Expected behavior
I think it would be more intuitive, if all results of one run are stored in the same time stamped folder.

Possible solutions
For subpipelines the path argument is ignored, instead they get the file manager of the containing pipeline.

@benHeid benHeid added the bug Something isn't working label Jun 22, 2021
@benHeid
Copy link
Collaborator Author

benHeid commented Jun 22, 2021

@olineumann do you have any further ideas or opinions?

@benHeid
Copy link
Collaborator Author

benHeid commented Jun 22, 2021

A problem would be if the same instance of a subpipeline is added to multiple other pipelines...

@olineumann
Copy link
Collaborator

olineumann commented Jun 22, 2021

I would expect that there are different directories for each subpipeline if each (sub-)pipeline has set a specific path. If the path of a subpipeline is None, I would like to have the option, that the path or FileManager of the parent Pipeline is used. Maybe by a flag in the train/test routine? pipeline.train(overwrite_none_paths=True) or pipeline.train(overwrite_subpipeline_paths=True)?

If we think about pipelines as modules in the future we could pass a path parameter to the __init__ method of the BasePipelineModule class? Would that solve this issue?

@benHeid
Copy link
Collaborator Author

benHeid commented Jun 22, 2021

I would expect that there are different directories for each subpipeline if each (sub-)pipeline has set a specific path. If the path of a subpipeline is None, I would like to have the option, that the path or FileManager of the parent Pipeline is used. Maybe by a flag in the train/test routine? pipeline.train(overwrite_none_paths=True) or pipeline.train(overwrite_subpipeline_paths=True)?

Probably this would be the most intuitive solution.

If we think about pipelines as modules in the future we could pass a path parameter to the init method of the BasePipelineModule class? Would that solve this issue?

Not sure, since we have to consider the time stamped folder created by the parent pipeline? I think problems may arise if the same instance of a subpipeline is used in two pipelines at the same time.. So the filemanager of the subpipeline would be the same for both parent pipelines. Consequently, in one case the time stamped folder of a parent pipeline would not be considered.
But perhaps, we should implement it and mention in the documentation, that such modules should only be used with care in multiple pipelines..

@benHeid benHeid transferred this issue from KIT-IAI/pyWATTS Dec 1, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants