Discuss: Credential Rotation & Unbind Semantics

[ransombriggs]

What is the problem?

We have been trying to figure out a way to implement credential rotation of individual bindings through the OSB specification. Additionally we need to support credential rotations on a schedule either initiated by the broker, or defined by the broker. This discussion has been ongoing and with previous discussions taking place in multiple issues / PRs.

• Binding Expiration here and here
• Bind / Unbind vs PATCH here and here

We have been discussing Bind / Unbind vs PATCH and would like some clarification around the following line in the specification.

When a Service Broker receives an unbind request from a Platform, it MUST delete any resources associated with the Service Binding. In the case where credentials were generated, this might result in requests to the Service Instance failing to authenticate.

This seems to indicate that when a binding is deleted that resources associated with the service binding should be deleted. Using multiple service bindings as a way to support credential rotation for all partners has the potential to be an unsafe assumption. We have been talking through various options to extending the OSB specification to allow for us to make credential rotation available to all service brokers in the platform.

We have sketched out how these steps could work but wanted to get some feedback around our assumption of possible data loss on unbind. We had discussed having another level of indirection between the platform and broker to mask this, which also seems complicate the user experience. Additionally we would need to save parameter bindings which might contain confidential information according to this paragraph, which while about instances, I assume may also apply to bindings.

Service Brokers MAY choose to not return some or all parameters when a Service Instance is fetched - for example, if it contains sensitive information.

Use-case: Datastore

A service binding represents a database role and the associated permissions. No user data is “owned” by the binding. Performing an unbind operation would just involve removing the role and leaving the data created using the role present in the database. This scenario has some ambiguity.

• If a bind / unbind approach is used, the platform must coordinate the credential rotation and the broker must support multiple bindings representing the same permissions.
• If a PATCH approach is used, the broker must track the multiple credentials inside the single binding and the associated permissions and support creating and revoking the credentials.

Use-case: Logging / Metrics

A service binding represents not only authentication and connection information, but also a stream of logs or metric data. Performing an unbind operation in these situations has some ambiguity.

• Should I mark records associated with this binding for removal or should the data remain after the binding removal?
• How and when should the data created using the service binding be deleted?
• If a bind / unbind approach is used, a broker would need an identifier to correlate the two bindings as a single data stream which is not available in a general manner currently.
• If the PATCH approach is used, a broker would need less context and only need to support creating and revoking credentials for a single binding.

Who does this affect?

This affects both service broker authors and platform authors.

Do you have any proposed solutions?

Depending on the discussion around this issue, we would develop a PR to outline the following protocol:

• Platform should notify broker that new credentials have been requested
• Broker supplies the platform with new credentials
• Platform notifies broker that old credentials are no longer needed

[mattmcneeney]

Hey @ransombriggs
Sorry for the slow feedback on this. Are you able to join the OSBAPI call today? I think it would be worth talking this through in more detail. Historically we've encouraged folks to simply create a new binding, and then delete the old one, giving the application time to make use of the new binding whilst both sets of credentials work. I think I understand the gist of this issue though.

This scenario stands out as one that would be hard to support today:

• The developer provisions a new MySQL database. This results in a new database being spun up on a new VM.
• The developer creates a new service binding. This results in a new table being created in the database with some credentials that can be used to access it.
• The developer wants to rotate the credentials as they were leaked.

Unbind/bind wouldn't work, as a service broker built to the spec would likely delete the table, resulting in the data being lost.

Another potential solution would be to allow bindings to be updated, with some flag that requests a new credentials object perhaps. I don't have a strong preference on how we'd solve for this, but I think it's important that credential rotation is triggered by the platform (as in CF, this requires an app to be restarted to pick up the changed credentials in the environment variable).

[ransombriggs]

@mattmcneeney the scenario you described matches the concerns that I have perfectly. I should be able to join the OSBAPI call today, see you then.

[gberche-orange]

@ransombriggs @mattmcneeney

Did the related discussion happened ? If so, can you please share the associated outcome (I did not manage to find it into the WG minutes) ?

[ransombriggs]

The #701 PR addresses most of my concerns. It makes it obvious that it is a continuation of a previous binding so that continuity can be preserved.