Secure communication with RE Manager using 0MQ API

[dmgav]

Implementation of options to enable secure communication between clients and RE Manager via 0MQ channel. Security is enabled by turning on Curve-based encryption, which is built in 0MQ library.

A key pair may be generated by using CLI command qserver-zmq-keys to generate new keys:

$ qserver-zmq-keys
====================================================================================
     ZMQ SECURITY: New public-private key pair.
====================================================================================
 Private key (RE Manager):                 kHAe2@EY5g>:w}OIYl5q.3Iry-8ew0cvr*h<EDWH
 Public key (CLI client or HTTP server):   NUXaxz1LO-h1TI>Bt^JJK9uYDhx&$:QJpjWt:e>k
====================================================================================

or a public key based on an existing private key:

$ qserver-zmq-keys --zmq-private-key 'kHAe2@EY5g>:w}OIYl5q.3Iry-8ew0cvr*h<EDWH'
====================================================================================
     ZMQ SECURITY: Private key generated based on provided private key.
====================================================================================
 Private key (RE Manager):                 kHAe2@EY5g>:w}OIYl5q.3Iry-8ew0cvr*h<EDWH
 Public key (CLI client or HTTP server):   NUXaxz1LO-h1TI>Bt^JJK9uYDhx&$:QJpjWt:e>k
====================================================================================

Once the keys are generated, encryption may be enabled for RE Manager by setting the environment variable:

export QSERVER_ZMQ_PRIVATE_KEY='kHAe2@EY5g>:w}OIYl5q.3Iry-8ew0cvr*h<EDWH'
start-re-manager

Encryption should be also enabled at the client side before client can communicate with RE Manager. All 0MQ API are now accepting the server public key as a parameter: constructors of classes ZMQCommSendAsync and ZMQCommSendThreads and function zmq_single_request accept additional server_public_key parameter. The developer using API is responsible for passing the public key to API classes and functions.

HTTP Server and qserver CLI clients are part of bluesky-queueserver package. Encryption for both clients can be enabled by setting QSERVER_ZMQ_PUBLIC_KEY environment variable before starting HTTP server or running qserver CLI. For example, HTTP Server can be started as

export QSERVER_ZMQ_PUBLIC_KEY='NUXaxz1LO-h1TI>Bt^JJK9uYDhx&$:QJpjWt:e>k'
uvicorn bluesky_queueserver.server.server:app --host localhost --port 60610

and qserver commands may be executed as

export QSERVER_ZMQ_PUBLIC_KEY='NUXaxz1LO-h1TI>Bt^JJK9uYDhx&$:QJpjWt:e>k'
qserver environment open
qserver queue start
qserver environment close

The PR addresses issue #146

[mrakitin]

Any ideas on how to fix the linter?

Error: Unable to resolve action `psf/black@stable`, unable to find version `stable`

[dmgav]

@danielballan looked at this issue. His opinion was that it will be fixed at certain point and we should just run 'black' manually.

[dmgav]

It happens to all our projects that use 'black'.

[mrakitin]

xref psf/black#2079

[mrakitin]

How about trying this solution meanwhile before the stable branch is reverted?

[dmgav]

I agree with the majority of participants of the discussion at psf/black#2079 who are not planning to do any changes. There are no external contributors to this project yet.

[mrakitin]

I like the idea of using a native encryption feature of 0MQ. I have concerns about passing the private key via CLI options but otherwise looks good.

[dmgav]

I think I applied all the recent suggestions in the latest commit. They are marked as outdated, but they are applied to the code.

[dmgav]

I also removed command-line options of setting encryption keys as potentially unsafe and also inconvenient. I also removed the description of the command-line options from the PR description.

[mrakitin]

Looks good. Suggested a few corrections, mostly in docstrings.