Task specific signature mode.

[BoPeng]

Right now, the tasks inherit signature mode from the sos instance. Even for PBS task engine, the signature mode is passed to the template and to the sos execute command.

I am wondering if there are cases that we want to keep signatures for main sos workflow, but use different signature mode for the tasks (something like task: sig_mode='ignore'). For example, the cluster might not like the multiple small files created by signatures (#991) and there is no need for signatures for large number of small tasks, or for production runs.

[gaow]

🥇 for the 1000^{th} issue ...

For example, the cluster might not like the multiple small files created by signatures (#991) and there is no need for signatures for large number of small tasks, or for production runs.

Sure that'd be very helpful! But how to prevent rerun / or trigger rerun in this case? Use some local .sos signatures?

[BoPeng]

Do not know. Need to think harder.

[BoPeng]

With the new .task file format, tasks store signatures within the completed .task file. There is therefore no longer a concern on small signature files. The only use of

task: sig_mode='ignore'

would be skipping the calculation of md5 of input and output files, at the cost of not able to skip a task, which is not important for production runs. However, I would argue in the later case a workflow level -s ignore would suffice so there is no need for task-specific signature mode.

So I think this will be WONTFIX unless there is another strong argument for it.