api-server-rest: Responded with 200 OK although providing error in the body

[davidhadas]

api-server-rest responds with "200 OK" with a body indicating a timeout error.

The body included:
rpc status: Status { code: INTERNAL, message: "[CDH] [ERROR]: Get Resource failed: get resource failed: get rersource failed: Kbs client error: get resource failed: http request failed: get failed: error sending request for url (http://127.0.0.1:8080/kbs/v0/resource/default/sshclient/publicKey): connection closed before message completed", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }

This is probably an example of a broader set of potential errors that lead to a "200 OK" with an error body.

When a. REST service reply with "200 OK" the REST client assumes that the body contains the requested information.
In this case, the REST client will assume that the body contains key material.

The REST client has no way to identify that an issue occurred and will only discover this later when trying to use the information as a key material and fail.

The api-server-rest should respond with a 4XX or 5XX to the client if the key material is not delivered.

The issue was observed when the connection between the CDH and KBS was abruptly disconnected during a key request.