Add support for shared steps

[xylar]

This merge adds support to the polaris framework for shared steps (steps that can belong to more than one task). Existing tasks and steps are modified to accommodate the changes to the framework but tasks and steps are not yet reorganized to take advantage of shared steps so that bit-for-bit comparison with main can still be made with this branch.

Changes to Step include:

• switch the subdir attribute to be relative to the component work directory, not the task work directory
• add an indir parameter to the constructor to easily construct a subdir that joins indir and name. If indir is the task work directory, this is the same as the old default behavior when subdir was not specified.

Changes to Component include:

• adding a dictionary of steps (with work-directory paths relative to the component's work directory as keys)
• add add_step() and remove_step() methods

Changes to Task include:

• modify the add_step() method to perform error checking (e.g. make sure a different step with the same path hasn't already been added) and then add the step to the component and the task. This method also includes a new symlink parameter that can be used to make a local symlink in the task's work directory to a shared step if it resides outside the task.
• add remove_step() methods

Other changes include:

• listing steps with full work-directory paths when polaris list --verbose is called
• removing tasks from step pickle files (since shared steps will not belong to a single task)
• add a polaris_step_complete.log file once a step has run to indicate that it is complete (and should not be run again)
• add --clean flags to polaris setup and polaris suite to remove the base work directory before setting up tasks. (This is sometimes useful when debugging because steps that have already run will now be skipped, rather than running again, which may not be what the developer wants.)
• adding a resolution_to_subdir() function to the ocean framework for conveniently converting a float resolution to a string.

Checklist
and changes look as expected

• Testing comment in the PR documents testing used to verify the changes

[xylar]

Testing

I have run all test cases except 1 and 4 km RPE tests and made sure results are BFB with main.

[xylar]

I don't plan on updating the documentation here. Instead, I will do that in #117

[xylar]

@cbegeman, thanks very much for the review. These are all great suggestions, which I will make tomorrow.

[xylar]

@cbegeman, I believe I've addressed your 3 requested changes. Please have another look when you have time.

[cbegeman]

@xylar I looked over the changes and they look good! Thanks for addressing all my comments so quickly. Let me know if you'd like me to do any additional testing.

[xylar]

@cbegeman, thanks for looking it over again. I think we're good for now. Let's see what @sbrus89 says.

[sbrus89]

Should this raise an error?

[xylar]

Perhaps this would be better?

Suggested change

	directory = os.path.dirname(os.path.abspath(link_name))
	try:
	os.makedirs(directory)
	except OSError:
	pass
	directory = os.path.dirname(os.path.abspath(link_name))
	try:
	os.makedirs(directory)
	except FileExistsError:
	pass

Something like this is the preferred way try to make a directory, but to do nothing if it already exists. It avoids a race condition that can happen if you first check if the directory exists and then create it.

[sbrus89]

Can steps be shared across components?

[xylar]

No, the current design is that steps belong one component only. You also cannot currently run tasks from different components in the same suite (or "custom" suite defined from a list of tasks), in part because the -p flag points to the directory where the model associated with the component exists.

[sbrus89]

I'm still al little fuzzy on the difference between subdir and indir could you explain it to me again?

[xylar]

The subdir is the directory under the components work directory where the step exists. The indir is one level up from that, with the name of the step appended to it to get the subdir. Before shared steps, the default behavior was to take the task's subdirectory and append the name of the step to get the default subdir for the step. The easiest way to efficiently recreate this default behavior without steps having ot know about tasks is to have indir be the task's subdirectory. But there's no reason that it has to be, it could be any convenient directory that is one up from the step's subdirectory, as long as it makes sense for the step's name to be appended to it to get the subdir for the step. I don't think there are currently any examples where indir is anything other than the parent task's subdir.

[xylar]

@sbrus89, thanks so much for your comments! I appreciate you taking a look. Let me know if I've addressed your comments or if you'd like to see more changes.

[sbrus89]

@xylar, thanks for the explainations! I sucessfully ran the nightly, pr, and cosine_bell suites.

[xylar]

Thanks so much!