Allow encrypted saved-object properties to be accessed by end-users

[kobelb]

Currently, encrypted saved-objects are intended to be used to store secrets which are specified by end-users, which can only be retrieved by the Kibana server itself. Fleet would like to store API Keys in encrypted saved-objects, which can then be retrieved by end-users.

We should add the ability for consumers of the encrypted saved-objects plugin to specify fields which should be accessible by end-users.

[elasticmachine]

Pinging @elastic/kibana-security (Team:Security)

[peterschretlen]

How is this different from encrypted saved objects?

[kobelb]

Currently, encrypted saved-objects are intended to be used to store secrets which are specified by end-users, which can only be retrieved by the Kibana server itself. Fleet would like to store API Keys in encrypted saved-objects, which can then be retrieved by end-users. The title is bad, and the lack of a description is even worse... fixing now.

[kobelb]

Hey @elastic/ingest-management when Fleet needs access to the encrypted API Key's key which is stored in the saved-object, how do you all feel about calling a dedicated API to get back the secret? Currently, the encrypted-saved-objects plugin strips all attributes which are encrypted from all operations. I began exploring what it'd take to return these attributes decrypted for all operations, and uncovered some interesting complications.

1. The decryption of these attributes can fail if the encryption key has changed or the AAD check fails. Surfacing this error for the operations which return a single saved-object is easy enough. However, we'd have to figure out how we'd want this to work for the operations which return multiple saved-objects, for example find/bulkGet. If we failed the entire operation when the decryption failed, it would rather drastically impact the end-user's experience. For example, we'd have to fail the operation to list all of the policies. We can also only fail the decryption for the single saved-object, and figure out how to return the error. The bulkGet already surfaces errors along these lines. However, the find operation does not. This complication can largely be overcome with enough effort.

2. Decrypting these attributes does consume CPU, I haven't performed any benchmarks to know exactly how much. However, not decrypting them does consume less CPU.

[kobelb]

@azasypkin I'd like to get your input on this as well. I might be overstating the complications. https://github.com/elastic/kibana/pull/57743/files has a super rough proof-of-concept, which doesn't handle the partial decrypt failures for the bulk operations.

[ruflin]

@nchaulet Could you chime in here?

[nchaulet]

@kobelb I feel okay with calling a different API, currently is already the case we call the decryptAsInternalUser method, I like the idea to select which attributes we want to decrypt.

For now we do not need encrypting|decrypting on bulk operations, but maybe we could optimize a few of our requests with that.

[roncohen]

as an aside, this is not just needed for Fleet, but will also be used for APM and for ingestion using regular filebeat/metricbeat/*beat.

[azasypkin]

as an aside, this is not just needed for Fleet, but will also be used for APM and for ingestion using regular filebeat/metricbeat/*beat.

While we're here, I hear about this requirement a bit here and there, but do we have any more or less formal documents/issues/RFCs with requirements and scenarios we'd like to support, and the long-term vision on this matter? It'd be super helpful if we can link something like this to this issue.

[roncohen]

ping @arisonl and @bytebilly around the formal requirement

[nchaulet]

To add a little more context and to be sure the things we are currently doing are working.
In fleet we use encrypted saved object to store api keys inside a saved object named EnrollementApiKeys. The enrollement key is created by an user action, and the user should be able to see the key again in the UI (only one at the time), the way we are currently doing it is using this method decryptAsInternalUser` in our API routes.

[azasypkin]

the way we are currently doing it is using this method decryptAsInternalUser in our API routes.

That doesn't sound ideal to me, but I guess we don't have other choice until we provide a better solution in the scope of this issue.

Btw, can you point me to the place in code where you do this? I couldn't find that in the master branch.

[legrego]

@arisonl do you have any updates to share? Last we talked, there was a discussion as to whether we should be doing this in Kibana via saved objects, or if Elasticsearch should be exposing a way for API Keys to be re-retrieved.

[nchaulet]

@azasypkin Sure we are doing it here https://github.com/elastic/kibana/blob/master/x-pack/plugins/ingest_manager/server/services/output.ts#L61

[arisonl]

@azasypkin @legrego what are the implications of the workaround Fleet is using? What are they losing? Multi-tenancy?

[arisonl]

@azasypkin @legrego As there is no change to the decision of making enrolment keys (and related functionality that may come up in contexts other than Fleet) retrievable by the end users through Kibana saved objects at this point in time, this remains a priority for 7.8.

[legrego]

@azasypkin @legrego what are the implications of the workaround Fleet is using? What are they losing? Multi-tenancy?

Yes, their current implementation lacks space-awareness, but that appears to be by design. They could implement this in a space-aware way today if they wanted to, but it would be a manual effort, as opposed to something we do automatically for them.

They're also missing out on audit logging by doing it this way. Currently, a user will call this API endpoint, and then the internal user will perform the retrieval and decryption of the saved object.

@azasypkin am I missing anything?

[azasypkin]

@azasypkin am I missing anything?

Nope, what you said already + it's a bit more fragile in a sense that there is a higher chance that the info that is only retrievable on behalf of the Kibana system user can be accidentally exposed to the user that doesn't have enough privileges for that info since defense line should be put in the application/route logic.