cli: display live logs

[jlemesh]

Related to reanahub/reana#824

reana-client logs command currently displays all available logs, with an option to use pagination (which seems to not work?). To retrieve newest logs, user would need to run the command multiple times. If all logs are returned at once, and with running the command multiple times, there would be a lot of duplicated command line output.

Create new command reana-client follow-logs that would continuously poll for new logs and append new data to the existing output, i. e. similar to tail -f. It would follow only one workflow or one job, not both and not multiple.

I think it's better to create new command instead of reusing reana-client logs because both commands would have different flags except for -w (there would be confusion regarding which flags affect which behavior), and code would look more complicated.

Usage: reana-client follow-logs [OPTIONS]

  Continuously poll for workflow or job logs.

  The ``follow-logs`` command allows to retrieve and follow logs of running
  workflow or job.

  If --step flag is supplied, command follows a specified job logs. If --step
  is not  supplied, command follows workflow logs.

  If workflow or job finishes running, the command exits.

  Examples:

       $ reana-client follow-logs -w myanalysis.42

       $ reana-client follow-logs -w myanalysis.42 --step 1st_step

Options:
  -w, --workflow TEXT      Name or UUID of the workflow. Overrides value of
                           REANA_WORKON environment variable.
  -t, --access-token TEXT  Access token of the current user.
  -s, --step TEXT          Step name to follow logs for.
  -i, --interval INTEGER   Sleep time in seconds between log polling.
                           [default=10]
  --help                   Show this message and exit.

Implementation: essentially just create a while True loop that requests logs data (either workflow or job), calculates a diff between previous and new logs and prints the diff. If workflow or job status becomes finished, failed, etc., stop the polling and exit. Otherwise sleep for --interval seconds and repeat.

Will need to implement for both Python and Go.

[tiborsimko]

Actually a new command could create confusion to the users by offering another choice. I think it's better to be clear towards users interface-wise, i.e. offering one logs command, and deal with any hidden implementation complexity (checking optional flag compatibility) on our end.

This would offer logs --follow which is similar to what people might be used to from tail -f, kubectl logs -f, etc. That was one of our design motives for ls, rm etc.

So I'd say let's rather implement reana-client logs -f and our function would deal with checking options internally, reporting that -f is not compatible with -x or some other option, as needed.

For example, regarding --step, we already have logs --filter step=gendata, so we could simply use that. And if people would like to follow and don't provide step, we can explain that this option is mandatory. (Until we implement later down the line some intelligent follow that would analyse current job, follow it, and when it finishes, follow another.... Even though that could leave some jobs out when many concurrent jobs are running, such a default follow might be already useful to some people with low job concurrency, and would work perfectly for mostly linear workflows. But that's something for later.)