Communications layer refactor, to RESTful HTTP

[hjoliver]

For network communication between cylc client and server programs, use HTTP(S) instead of Pyro.

This has been planned for a long time, but we didn't have an issue up for up it.

• Replace obsolete Pyro-3 Object RPC .
• Support a stable RESTful HTTP API for interaction with suite servers (not tied to Python for this); HTTP(S) is these days the best protocol for talking across ports
• Will enable (because the suite IS a web server) us to leverage a lot of existing web-based technology, e.g.,
  • web based suite GUI (also PyGTK is dead)
  • better authentication (use of existing technologies)

[trwhitcomb]

Glad to see there's an issue up so I can get this captured somewhere. I'd like to broaden the scope of this a bit to have a generic communications layer refactoring, with the RESTful HTTP as the default (and preferred) option, but with the facility for defining a standard way of implementing a communications API that can take advantage of other messaging solutions (e.g. something like ZeroMQ, cloud-based messaging platforms, etc.). This would make it easier to handle situations where the connection setup between computers isn't always ideal, and perhaps bypass other issues (e.g. security might object to a "web server")

[hjoliver]

@trwhitcomb - I agree in principle, but I can't see the current development team having the resource to implement multiple communications layers (and my impression is security people would sooner accept HTTP than some less common or custom protocol). I guess a stable API would make it possible though, if (for example) you wanted to do the work 😁

[trwhitcomb]

@hjoliver - to clarify, I'm not currently asking for the current team's implementation of any other communications layers, but I think it would be useful instead of specifying a single method to have an API for performing the communication that users (e.g. me) could code to to allow other communication methods instead of the "standard" HTTP-based method. That way if I need to work around something, I can start from a specification rather than undoing what's already hard-coded into the software.

And yes, you would think that about security people but you might be surprised!

[hjoliver]

[meeting]

• maybe handle task messaging differently from user interaction: task authentication can be automatic (suite gives tasks the credentials?)
• for cylc-7 (major version up, but little impact on users)
• remove all back-compat code for client server interaction across versions, along with this change, as we cannot maintain this compatibility anyway for this change.
• @benfitzpatrick already has a working development branch, just ~10 failing tests to investigate.