Detached payloads can be used

[selfissued]

Fixes #278

Also updated the SD-JWT terminology to use the term "JWT Claims Set", which the SD-JWT spec now does.

---

Preview | Diff

[msporny]

The WG needs to discuss this. This would be the first time we're doing a fully detached payload for a VC, with no reference to the detached payload anywhere in the signed information. The use case isn't clear, and it seems like a dangerous thing to do (you're just signing something and then not giving any guidance on where that thing lives or how to use it in a protocol).

[selfissued]

I'm fine with the working group discussing it, but the underlying facts of the situation are that both JWS and COSE support detached payloads. Therefore, this PR is editorial, in that it doesn't change the normative meaning of the specification or add any functionality that didn't already exist.

What it does do is remove any ambiguity that may be in people's minds about this functionality already existing and being usable with Verifiable Credentials.

As for security, there's nothing less secure about signing detached content than attached content. In both cases, you continue processing the input only if the signature validates.

[iherman]

The issue was discussed in a meeting on 2024-08-21

• no resolutions were taken

View the transcript

5. Detached payloads can be used (pr vc-jose-cose#292)

See github pull request vc-jose-cose#292.

Gabe Cohen: It came to my attention that there's a PR in JOSE-COSE that we should discuss.

Michael Jones: Ambiguity around using detached signatures. If we do nothing, we can use detached signatures. That said, readers would be better served for us to repeat what's in the underlying specifications. It's not normative, because it doesn't change the meaning of the spec.

Manu Sporny: The reason this surprised me is because I don't think we use detached signatures anywhere else in the ecosystem.
… We see a very large payload followed by a very small signature payload without reference to the original.
… I don't understand what the use case for that is.
… If we're going to support this, we probably need to talk about how that signature is associated with the payload.
… It feels like it could lead to a number of security concerns. I want to know what the concrete use case is.
… If there is a valid use case, then we need to make sure that we have thought about the implications of just doing a detached signature. Are we going to provide any guidance at all?

Michael Jones: I also answered the security question in the issue that it's no less secure than including it in the payload, because if it doesn't validate, detached or not, we can't proceed. How do we associate the payload with the container? We don't do protocols, and that's a protocol feature.
… There are plenty of cases in the real world where there is data in various channels and you want to validate that it's correct. It's out of scope for us to say how a signature is transmitted or checked against a piece of data.

Joe Andrieu: In the way that we're using the detached payload, what's going over the wire has a media type of application/vc, but because it's detached, it's not going with the payload. What do we call that thing that's securing that data, that is not a VC?

Ivan Herman: Without going technical, it's a procedural thing. My understanding is that the fact of having the detached signature is something that the standard allows.
… If we do not want that, we have to actively in the standard disallow it, which I think would be a heavy thing from our side to do. It's out there, it's the user's business to make sure it's secure, it's not our responsibility.

Michael Jones: That's correct, and likewise COSE defines the mechanism for detached signature.

Manu Sporny: I haven't yet heard a use case where we need to use detached signatures. It's really hard to analyze the usefulness until we have them. I have concerns, because in our COSE examples, we're going to be showing how to secure things, and we will be showing detached signatures, but we don't have a media type for it and we don't know how to.

Kevin Dean: associate it with the original payload.

Manu Sporny: There are a number of questions. We shouldn't put a number of examples if the group isn't using it. Others can use it and other mechanisms, but we don't have examples in the spec because we haven't got concrete use cases.
… I'm fine with staying silent on it, not with putting examples throughout the specifications.

Michael Jones: This PR doesn't change the examples, nor am I proposing to do so. One of the technical reasons in JOSE for using detached signatures is that you can use the unencoded payload option, which means you never have to Base64 encode the payload. If you have an authenticated way to transmit the payload between parties, that's outside the scope of what we specify. We get the size savings of not doing the encoding. That is a use case.

Joe Andrieu: seems like Mike's payload is not a VC, as in application/vc, but something else.

Manu Sporny: I would be find with us staying silent on it. Someone out there that is implementing this would be good to hear from.
… It raises a whole bunch of other questions that have not been answered.

Gabe Cohen: We do use detached payloads. We haven't explored COSE yet, but will ask my team.

[decentralgabe]

since JOSE and COSE support detached payloads it makes sense to me that this spec should support them too

noting it may be worth it to add guidance on when it would be appropriate to leveraged detached payloads. tagging @mistermoe to speak to our usage of detached payloads at TBD.

[decentralgabe]

After speaking with @mistermoe, I can reaffirm our usage of detached payloads. In our project, tbDEX, our signatures currently make use of detached JWS.

Why detached JWS? Detached signatures are employed to facilitate scenarios where the payload (i.e., the data being signed) is already available or transmitted separately. By using a detached signature, the original payload remains unaltered and can be transmitted or stored in its native format without being embedded in the signature itself. This approach avoids redundancy but also allows recipients to independently verify the integrity and authenticity of the payload using the detached signature and the signer's public key.

In addition to the reason given in the docs, we find efficiencies storing detached signatures in our databases. We are able to store the payload data in queryable text, while maintaining the signature. As necessary, we're able to reconstruct the detached payload and verify the signature. This is preferable to storing the data twice (in plaintext and base64).

We have not yet explored adopting COSE; however, it's a topic of interest within the team, to provide additional space savings.

That said, we would like to see this specification support detached payloads for both JOSE and COSE.

[iherman]

The issue was discussed in a meeting on 2024-09-27

• no resolutions were taken

View the transcript

2.3. Detached payloads can be used (pr vc-jose-cose#292)

See github pull request vc-jose-cose#292.

Brent Zundel: PR raised by selfissued, comments by decentralgabe,.

Gabe Cohen: spec was unclear about detached payloads, this PR adds that language,.
… they allow for detached payloads, so should we.
… my company makes use of detached payloads, it's more efficient to store the signature separately.

Michael Jones: as discussed in the issue, the jwt spec and the cose spec permit detached payloads and we are using them normatively, if we do nothing they are allowed, if we merge this PR it doesn't change the meaning but it does explicitly say detached payloads are allowed.
… ok to close or merge, would object to change that would try to inhibit the use.

Manu Sporny: this came about because the default example changed, will note that I oppose this because it's the first time we're talking about using detached payloads for VCs.
… I have an opinion but want to see what the group has to say.

Mandy Venables: as selfissued said, if it's allowed then it's allowed, and find decentralgabe use case compelling.

Manu Sporny: concerned about hexadecimal signature in example, would rather follow the same format as other examples.
… waiting on feedback from decentralgabe.

Gabe Cohen: feel like there was regression in code, example should show full payload, it may be useful to show examples for attached and detached.

Manu Sporny: concerned about detached payloads were viewed negatively and used as an example of the group not knowing what it is doing.
… not technical concern, but do we want to invite criticism back in?
… incubate in a group. Alternative is to write it up now, but concern without implementations.

Brent Zundel: proposal is to close this PR now, raise another one where the examples are only attached payloads.

Manu Sporny: clear up the example to show full payloads.

Michael Jones: if we close the PR we should close the issue.
… if we get other criticism about detached payloads I will take it on, they fail safe because of crypto.

Brent Zundel: but you are not disagreeing.

Michael Jones: yes.

Dave Longley: +1 detached payloads are fine, signatures fail safe if the payload doesn't match.

Brent Zundel: anyone who would disagree about closing this PR and issue?
… no opposistion.
… let's get started, we will start by talking about controller document.

[decentralgabe]

As per the TPAC discussion, we are closing this as 'won't fix'. Detached payloads are possible, and we do not need language around them.