Support disabling working directory management

[omry]

Disabling the output directory management should be supported.
This includes:

• Not changing the working directory
• Not saving the .hydra directory
• Potentially not saving a log file

Workarounds in 1.0 / 1.1:

• Not changing the working directory: override hydra.run.dir to .
• Not saving the .hydra directory: override hydra.output_subdir to null
• Potentially not saving a log file: override hydra/job_logging to stdout.

[benjamindkilleen]

Perhaps there is a less drastic option than disabling these capabilities entirely, without changing the cwd, that could be the default behavior. Hydra could still capture logs and configs and store those in the output dir. At the same time, giving the program access to this through hydra.get_output_dir() or hydra.output_dir would still allow the user to write to this directory while still assuming a non-changing cwd.

This would be more natural for new users, who are probably confused when their program can't find a data file on a relative path, while still providing them with unobtrusive logging storage and access to the output dir when they've become more familiar with hydra (there are a lot of docs to read and understand).

[omry]

You don't seem to understand the proposal here.
This is not about removing any feature, but about allowing people that does not want the feature to easily not have it. (regardless of what the exact user experience is).

The idea is to support disabling it, not to kill the feature.
Enabling different control over chdir (while still generating an output directory) is definitely an option here.

[lkhphuc]

This would be more natural for new users, who are probably confused when their program can't find a data file on a relative path, while still providing them with unobtrusive logging storage and access to the output dir when they've become more familiar with hydra (there are a lot of docs to read and understand).

I know it's not related to this issue but I quite like this behavior, instead of the reverse currently.
Default to the path user is on, which is usually the path user provided from outside the code, use hydra.output_dir() inside the code programmatically where necessary (pass to Tensorboard logdir, Checkpoint callback etc...)

[kumuji]

Maybe I got up on the wrong side of the bed today, I am really sorry if it sounded angry, I didn't mean anything bad =(

I see that package is developing good, and it definitely requires a lot of effort to maintain it and make it for a lot of people. You discussed here an opportunity to better manage disabling of output logging, and I thought moving it as plugin might be a good option. But I guess it sounded like an unwanted advice and complaint about developing direction choices, and I conveyed my message wrong. Sorry, I apologize if I offended you in any way.

[omry]

Apology accepted. I understand this can be frustrating.
The reason Hydra is configuring logging is related to parameter sweeps and remote execution.
When you run the same app multiple times you want to allow each instance to log to different file.

Check out this pull request (merged into master), it can show you how to disable logging.
You can do exactly the same from your own application config. If you have question let's move them off this issue because this is not related to it (you can ask in the chat or open a separate question issue).

[omry]

This will not happen for 1.1, sorry.
Changing this is a major change that should be considered in the context of other breaking changes, it will probably require a redesign of the output directory management.

[nielstenboom]

For folks landing here wanting to disable the automatic changedir:

https://hydra.cc/docs/tutorials/basic/running_your_app/working_directory/#disable-changing-current-working-dir-to-jobs-output-dir