"=" in the checkpoint name may be incompatible with OSs

[Nic-Ma]

❓ Questions/Help/Support

Issue raised by Eric from MONAI team:
File names with operators in them may be valid names in Linux but that's not the case in other OSs, plus shells like bash really don't like them.

[sdesrozis]

@Nic-Ma Thank you for this question.

I don't know any problematic OS with = operator in filenames but even with Linux, you have to be careful and sometimes use quotes.

The filename policy is here

ignite/ignite/handlers/checkpoint.py

Lines 93 to 105 in cc78b3b

	By default, none of `score_function`, `score_name`, `global_step_transform` is defined, then suffix is
	setup by attached engine's current iteration. The filename will be
	`{filename_prefix}_{name}_{engine.state.iteration}.{ext}`.
	If defined a `score_function`, but without `score_name`, then suffix is defined by provided score.
	The filename will be `{filename_prefix}_{name}_{global_step}_{score}.pt`.
	If defined `score_function` and `score_name`, then the filename will
	be `{filename_prefix}_{name}_{score_name}={score}.{ext}`. If `global_step_transform` is provided, then
	the filename will be `{filename_prefix}_{name}_{global_step}_{score_name}={score}.{ext}`
	For example, `score_name="neg_val_loss"` and `score_function` that returns `-loss` (as objects with highest
	scores will be retained), then saved filename will be `{filename_prefix}_{name}_neg_val_loss=-0.1234.pt`.

Maybe we could add a format based on regex that user could set (default is existing policy) ?

[Nic-Ma]

Hi @sdesrozis ,

Thanks for your quick response.
A regex or user-defined separator sounds good, the default still can be "=".
Thanks.

[Joel-hanson]

Shall I take this up @sdesrozis

[sdesrozis]

@Joel-hanson thank you to contribute on this issue 😊

The idea is to replace the operator = in filename of checkpoints. To do it, we can introduce a separator attribute (default is =) as suggested by @Nic-Ma. A more sophisticated option is to provide a string pattern like {filename_prefix}_{name}_{score_name}={score}.{ext} and process it by replacing labels.

What do you think?

[vfdev-5]

How about setting this pattern as a class member of Checkpoint:

class Checkpoint:

     filename_pattern="{filename_prefix}_{name}_{score_name}={score}.{ext}"
     
     ...

If user would like to change it, there is a possibliity.

[Joel-hanson]

@sdesrozis replacing labels seems like a good option and the suggestion by @vfdev-5 is worth considering. I will note down what I have in mind to be more clear before moving forward.

I will set the filename_pattern="{filename_prefix}_{name}_{score_name}={score}.{ext}" as a class member for the Checkpoint. The filename will have the default pattern if there is no change in the class member. If a user changes his filename_pattern, then by using str.format_map I will replace each label with its values.

When we implement likewise whenever a user provides the pattern to be {filename_prefix}_{name}_{score_name}, without having the either {score} or {ext} shall I append the default pattern to it like {filename_prefix}_{name}_{score_name}{score}.{ext} so that the file doesn't get replaced or not get saved without extension. Is there anything that I am missing out on? correct me if I am wrong.

[sdesrozis]

@Joel-hanson What you have in mind seems good 😉 str + format with fixed keywords (score, ext, etc.) looks good. It’s similar to what is done in logging format. It should be a good inspiration.

And you are right, we have multiple patterns depending of the usage context. IMO if user give a pattern, we should use it. Otherwise, legacy policy remains used. Let’s see if it fits class member approach ! What do you think ?

[vfdev-5]

When we implement likewise whenever a user provides the pattern to be {filename_prefix}{name}{score_name}, without having the either {score} or {ext} shall I append the default pattern to it like {filename_prefix}{name}{score_name}{score}.{ext} so that the file doesn't get replaced or not get saved without extension.

@Joel-hanson let's consider that if user changes the pattern he/she knows what he/she is doing.
So, no internal additional appends. For example, if pattern is

p = "{name}_{score}"
kw = {"name": "abc", "filename_prefix": "pref", "score_name":"acc", "score": 0.234, "ext": "pt"}
p.format(**kw)
> 'abc_0.234'

[Joel-hanson]

Thanks, @sdesrozis @vfdev-5 for the quick response. So I shall start implementing based on what @vfdev-5 has conveyed.