custom error messages from tasks

[mlin]

When a task is going to fail, it'd often be helpful to communicate a succinct error message, to be picked up by the engine and ultimately communicated to the user. For example, "Too few reads passed filters." If the task doesn't generate a custom message then the engine could substitute a generic suggestion to read the standard error log.

Here's an example hack we've done with miniwdl...it's by no means perfect, so I don't mean this as a concrete proposal but just a discussion starting point. https://github.com/chanzuckerberg/miniwdl/blob/main/examples/run_with_custom_error.py

[patmagee]

This is interesting, and I have seen this creep up multiple times in the slack channel. In general, I like the idea of giving wdl users a mechanism to provide better failure information to the engine so that they can diagnose issues more easily.

Your proposal is a good one, although I think it likely would not suite a lot of tasks that are very chatty in the stderr. Printing a formatted text to stderr would require the engine to parse and interpolate the stderr file in order to get the failure message. This obviously is not optimal since some tools write a heck of a lot of information to stderr.

What might be a better approach would be one of the following:

1. Engine function for supplying a task failure message. This might be challenging since it would be called during a task execution, and not prior to the task execution. So the engine function would actually no work here...

2. Define a special file handle which the user can write text out to which would be considered the error message. This could be formatted as json like the proposal which you put together, or simply a text file. A dedicated file would allow the user to not worry about what the tasks out to, and would optimize the engines handling:

version 1.0
task test {
    input {
        Boolean fail
    }
    command <<<
        >&2 echo 'Hello!!!! }{}"'
        if [ "~{fail}" == "true" ]; then
            echo '{"wdl_error_message": "this is the end, my only friend, the end", "meaning": 420}' > wdl_failure_message.json
            exit 42
        fi
    >>>
}

we can even go as far as describing in the runtime section a failureMessageFile. I am not exactly sure how this would all map onto some execution backends like TES tbh, but I wouold hope it could be supported if its file based.

version 1.0
task test {
    input {
        Boolean fail
    }
    command <<<
        >&2 echo 'Hello!!!! }{}"'
        if [ "~{fail}" == "true" ]; then
            echo '{"wdl_error_message": "this is the end, my only friend, the end", "meaning": 420}' > wdl_failure_message.json
            exit 42
        fi
    >>>

   runtime {
     failureMessageFile: "wdl_failure_message.json"
   }
}

[jdidion]

Yet another option is to define an engine function for this:

try("bwa mem ~{input_fa}", "error while aligning ~{input_fa}")

The engine would be responsible for converting this into the correct bash when evaluating the command block. For example:

output=$(bwa mem myinput.fa)
rc=$?
if [[ $rc != 0 ]]; then
  echo {"code": $rc, "message": "error while aligning myinput.fa"} > wdl_error.json
fi

This has the benefit that the engine can handle the error however it wants, so long as it adheres to some specification for reporting the error to the user. I like the idea of standardizing an error format in the same way we standardize an input and an output format.

[jdidion]

Of course multi-line strings would make it much nicer to write a long command as a function argument.