remove run and work dir configurations

[oliver-sanders]

Remove:

• global.cylc[platforms][<platform name>]run directory
• global.cylc[platforms][<platform name>]work directory

These have been made obsolete by the global.cylc[symlink dirs] configurations.

(We now install all flows into ~/cylc-run/<flow-name>, creating symlinks to locations on other filesystems where $HOME is not suitable).

Pull requests welcome!

[dpmatthews]

After looking again at #2779 and metomi/rose#2252 I fear there is still a requirement for run directory.

[oliver-sanders]

$HOME on the SSH remote is not the same $HOME as on the execution host.

If I've read them correctly these issues relate to $HOME on login nodes not being the same as $HOME on compute nodes which makes sense to me and is probably quite common.

If we can assume the compute nodes can see the login node $HOME filesystem and that it is mounted the same way on both nodes then we could potentially just expand $HOME on the login node before job submission.

This would also address the cases where $HOME is not set on compute nodes.

[oliver-sanders]

Dumping the result of a conversion with @dpmatthews on the matter here so we don't forget.

The remaining issues relate to $HOME differing between the submission and compute nodes on certain job hosts.

The new symlink dirs functionality could potentially provide a way forward here for cases where the login node can see the $HOME filesystem of the compute node, providing that this does not differ between compute nodes accessible from the same login node:

login-node:
  /home/
    login/
      me/ ($HOME)
       cylc-run/
         my-suite -> /home/compute/me/cylc-run/my-suite
compute-node:
  /home/
    compute/
      me/ ($HOME)
       cylc-run/
         my-suite

[dpmatthews]

I think we should go ahead and remove the existing settings (they don't make sense as platform settings in any case). If we find there is missing functionality that requires a new setting this can be addressed later.

[MetRonnie]

Just below is global.cylc[platforms][<platform name>]suite definition directory, which doesn't have a description. What is it, and can it be got rid of?

[dpmatthews]

I think that's probably a mistake and needs removing. Is it used anywhere?

[MetRonnie]

cylc-flow/cylc/flow/task_job_mgr.py

Line 1055 in c45aa27

'remote_suite_d': itask.platform['suite definition directory'],

@wxtim Do you know about it? I see in your PR #4126

Remove, and add obseletion warning for config item flow.cylc:[runtime][__TASK__][remote]suite definition directory (But not the similar platform item from global.cylc).

[wxtim]

I have no idea why there was an exception for the platforms item. Does it allow the suite definition dir to change by platform? Seems odd, and like it is a mistake.

[MetRonnie]

The only real reference to the global suite definition directory is

cylc-flow/cylc/flow/task_job_mgr.py

Line 1055 in c45aa27

'remote_suite_d': itask.platform['suite definition directory'],

And I can't find any references to remote_suite_d other than unit tests

[MetRonnie]

Also, the config change proposal says

(Note that we also need to deprecate the environment variables $CYLC_SUITE_DEF_PATH & $CYLC_SUITE_DEF_PATH_ON_SUITE_HOST.)

which suggests the global item should go too

[hjoliver]

I believe "remote suite definition directory" is an ancient thing that refers to where the user manually installed suite source files on the remote job host. Not needed any more.

[MetRonnie]

Should the --run-dir option of cylc suite-state be removed? I can't see any reason to keep it

cylc-flow/cylc/flow/scripts/suite_state.py

Lines 148 to 153 in 4f4db80

	parser.add_option(
	"-d", "--run-dir",
	help="The top level cylc run directory if non-standard. The "
	"database should be DIR/REG/log/db. Use to interrogate "
	"suites owned by others, etc.; see note above.",
	metavar="DIR", action="store", dest="run_dir", default=None)

While I'm at it, I'm guessing we can remove this obsolete option altogether?

cylc-flow/cylc/flow/scripts/suite_state.py

Lines 143 to 146 in 4f4db80

	parser.add_option(
	"--template", metavar="TEMPLATE",
	help="Remote cyclepoint template (IGNORED - this is now determined "
	"automatically).", action="store", dest="template")

[hjoliver]

Should the --run-dir option of cylc suite-state be removed? I can't see any reason to keep it

Still needed to interrogate other-user DBs, no?

While I'm at it, I'm guessing we can remove this obsolete option altogether?

👍

[MetRonnie]

Still needed to interrogate other-user DBs, no?

Clearly I am blind 🤪