Unify design of status lists to support multi-bit values

[msporny]

PR #65 added a backwards-compatible mechanism for binary status lists and multi-value status lists. During the discussion of the PR, there were multiple concerns raised around ttl, verification and validation and how the status list might affect those processes, the data formats utilized, the differences between binary and multiple bits per status. It was suggested that perhaps a re-alignment and unification of the design to support the use cases more generally would be useful. This issue tracks that design discussion.

[msporny]

Starting us off with a proposed unified design, example first:

{
  "id": "https://example.com/status/3#list",
  "type": "StatusList2021",
  "statusField": [{
    "statusBitOffset": 0, 
    "statusPurpose": "revoked",
    "activeStatusMessage": "This credential has been revoked by the issuer.",
    "inactiveStatusMessage": "This credential has been revoked by the issuer."
  }, {
    "statusBitOffset": 1, 
    "statusPurpose": "suspended",
    "activeStatusMessage": "This credential has been suspended by the issuer.",
    "inactiveStatusMessage": "This credential has not been suspended by the issuer."
  }, {
    "statusBitOffset": 7, 
    "statusPurpose": "recalled",
    "activeStatusMessage": "This credential has been recalled due to a health concern.",
    "inactiveStatusMessage": "This credential has not been recalled."
  },
      ...
  ],
  "encodedList": "H4sIAAAAAAAAA-3BMQEAAADAAAAAAAAAAAAAAAAAAAAAAAAAAAIC3AYbSVKsAQAAA"
}

• Establishes a new statusField property, which contains the information for each field in the bitstring associated with each entry (including messages -- this is somewhat analogous to statusMessages in PR Support for multiple status codes #65).
• Moves statusPurpose into statusField.
• Moves away from hexadecimal notation for status bit position to just a plain integer offset (so we can avoid hex to integer conversion... since we're trying to get to an integer bit offset anyway).
• Adds messages for active and inactive bits.
• Removes the size parameter, as it's not needed (you just pick the max statusBitOffset and add one to it's value to get the size.
• Removes reference; do we need it anymore if there is a description for every field?
• Removes ttl; can we get away with just validUntil?
• Removes the need for undefined fields... just leave a gap in your bit offsets if you want to leave fields undefined.

Questions that are left unanswered:

• How is verification and validation affected by the above?
• We need to write how having multiple bits per status negatively impacts privacy by enabling new correlation vectors?
• There are now i18n considerations due to the status messages. JSON-LD can provide an answer on multi-lingual expressions, BUT, the feature is rarely (if ever) used. I expect that the language expressed in the messages is typically in the language of the issuer.

Thoughts on this unified approach, @mprorock, @OR13, @dlongley?

[OR13]

Why use literal strings for
statusPurpose?

Seems like a URL would be better, and you could put the active and inactive language at the resource the URL dereferences too.

Let's get rid of all the sugar and add back only what needs to be understood by a verifier.

[msporny]

Why use literal strings for statusPurpose? Seems like a URL would be better, and you could put the active and inactive language at the resource the URL dereferences too.

Yes, I had considered that approach, but felt folks might reject the direction as being "too Linked Data-y". If we go that route, would there be standard URLs for revocation and suspension? (I'd presume so).

The messages might be useful for verifier platforms, if they wanted to elaborate on all of the statuses associated with credential. I thought traceability had a need for that, if not, we might want to simplify here.

[mprorock]

some inline comments below

• Establishes a new statusField property, which contains the information for each field in the bitstring associated with each entry (including messages -- this is somewhat analogous to statusMessages in PR Support for multiple status codes #65).
• Moves statusPurpose into statusField.

This is just naming - easy enough for implementers (ourself and others) to adjust naming if the WG goes this direction.

• Moves away from hexadecimal notation for status bit position to just a plain integer offset (so we can avoid hex to

integer conversion... since we're trying to get to an integer bit offset anyway).
that seems odd to me, we are looking at a bit backed structure and hex is a pretty normal for saying these bits mean x

• Adds messages for active and inactive bits.

cool

• Removes the size parameter, as it's not needed (you just pick the max statusBitOffset and add one to it's value to get the size.

definite size parameter feels like the right way rather than looking inside, looking for the largest number, and then adding one

• Removes reference; do we need it anymore if there is a description for every field?

yes - we likely want to be able to point directly back to regulatory references etc

• Removes ttl; can we get away with just validUntil?

no - ttl refers to the client (or cdn) question of "how often to i have to get a fresh copy of this thing?" - vaidUntil is when a new credential must be issued by the issuer

• Removes the need for undefined fields... just leave a gap in your bit offsets if you want to leave fields undefined.

alot of this would be much easier/better in cbor

[dlongley]

@mprorock,

no - ttl refers to the client (or cdn) question of "how often to i have to get a fresh copy of this thing?" - validUntil is when a new credential must be issued by the issuer

My understanding is that the goal of ttl is to enable issuers to avoid having to reissue status list VCs more frequently. If I understand the aim here, I don't think it actually works in practice.

If you sign a VC with a validUntil of sometime "next year" and a TTL of 5 minutes, you are indicating that some verifier somewhere may accept whatever was in that VC (with your signature on it) anytime between when that VC was first accessible and "next year". The TTL doesn't change that; it isn't helpful in this regard and there is no reason why the verifier couldn't just ignore it.

Additionally, consider one of the use cases for StatusList2021, where a holder fetches a status list VC and presents it along with whatever other VC is in the list it describes (in order to avoid having the verifier even contact the issuer at all). That holder can completely ignore the TTL and the verifier will be receiving it "within the TTL" when the holder presents it. There's no signature over when the holder requested it that could be checked, etc. Even if there were -- this would upend the goal of ttl by requiring more frequent signatures again.

In this scenario, an issuer might be thinking that if they always set a ttl of 5 minutes in their status list VCs then they will only have to wait 5 minutes after they publish a new status list VC to ensure it's the one all the verifiers use. They would be mistaken. This is based on an erroneous understanding of the meaning of the data and how the data and the various decentralized systems interacting with it are partitioned.

So, right now, I don't think ttl achieves its goal and therefore ends up adding both unnecessary complexity and a potential violation of expectations.

An issuer should instead decide on a validity period they are willing to commit to (maybe it's 5 minutes, maybe 15 -- maybe 24 hours -- maybe more, their call) and reissue the status list VC with an updated validUntil either on demand or on schedule. Then a verifier just checks validUntil per usual and can safely cache the status list VC as long as it's valid.

So, as an issuer, if you want to allow your status list VCs to expire every 5 minutes, issue a new one if any VC status changes (understanding that the status won't be consistent until after the old status list VC expires) and auto-reissue when a status list VC is requested if it has expired, bumping validUntil 5 minutes into the future each time.

Regarding privacy-preserving but consistent status tracking for newly issued VCs, do not initialize your status list bits as scrambled, but assign from a pseudo-randomly selected index or slice of the list. If some kind of scrambling or other privacy-preserving setting of the bits is still somehow required, ensure that there are always some unallocated areas of the list that match what a newly issued VC status would be -- and choose from those sections of the list for any new VC, simultaneously preparing any other unallocated areas for the next one or more VCs to be issued in the future. In other words, ensure that over the next "validity period", there is always enough "prepared, unallocated space" for the maximum number of VCs you could issue over that period.

[samuelmr]

The Working Draft links to this issue in a note that seeks feedback.

I read #65 and #47 and wondered if the following syntax has been suggested or considered:

{
  "@context": [
    "https://www.w3.org/ns/credentials/v2"
  ],
  "id": "https://example.com/credentials/status/3",
  "type": ["VerifiableCredential", "BitstringStatusListCredential"],
  "issuer": "did:example:12345",
  "validFrom": "2021-04-05T14:27:40Z",
  "credentialSubject": {
    "id": "https://example.com/status/3#list",
    "type": "BitstringStatusList",
    "status": [
      {
        "statusPurpose": "revocation",
        "encodedList": "H4sIAAAAAAAAA-3BMQEAAADCoPVPbQwfoAAAAAAAAAAAAAAAAAAAAIC3AYbSVKsAQAAA"
      },
      {
        "statusPurpose": "suspension",
        "encodedList": "H4sIAAAAAAAAA-3BMQEAAADCoPVPbQwfoAAAAAAAAAAAAAAAAAAAAIC3AYbSVKsAQAAA"
      },
      {
        "statusPurpose": "pending_review",
        "encodedList": "H4sIAAAAAAAAA-3BMQEAAADCoPVPbQwfoAAAAAAAAAAAAAAAAAAAAIC3AYbSVKsAQAAA"
      }
    ]
  },
  "proof": { ... }
}

IMHO, that would be an easy way for the verifier to get a bunch of status values about a credential. After parsing, they might end up with something like

{
  "id": "https://example.com/credentials/status/3#94567",
  "revocation": false,
  "suspension": false,
  "pending_review": true
}

That should be relatively easy to act upon.

The statusPurpose of a VC should be changed to an array:

{
  "@context": [
    "https://www.w3.org/ns/credentials/v2"
  ],
  "id": "https://example.com/credentials/23894672394",
  "type": ["VerifiableCredential"],
  "issuer": "did:example:12345",
  "validFrom": "2021-04-05T14:27:42Z",
  "credentialStatus": [
    {
      "id": "https://example.com/credentials/status/3#94567",
      "type": "BitstringStatusListEntry",
      "statusPurpose": ["revocation", "suspension", "pending_review"],
      "statusListIndex": "94567",
      "statusListCredential": "https://example.com/credentials/status/3"
    }
  ],
  "credentialSubject": {
    "id": "did:example:6789",
    "type": "Person"
  },
  "proof": { ... }
}

[msporny]

@samuelmr wrote:

I read #65 and #47 and wondered if the following syntax has been suggested or considered:

Yes, we had considered that, but ended up with the following (more composable) construct in the spec today:

#37 (comment)

The statusPurpose of a VC should be changed to an array

That would be an incorrect way to model the data. Effectively, what you're proposing would say that the bit that handles "revocation", "suspension", and "pending_review" is bit 94567, which would be wrong (because flipping the bit would flip all 3 states). An alternate reading that presumes bit offsets based on the three items listed would prevent bit packing, which was a requirement from a subset of the group.

I believe the current specification, including the pending PRs, will enable the use case you were attempting to address in a more composable way, and in a way that supports bitpacking.

[msporny]

At present, there doesn't seem to be enough desire to rewrite/unify the design to support multibit values. The current specification achieves all of the use cases outlined and is "good enough" to enter Candidate Recommendation.

I'm marking this issue as "pending 7 day close". Feel free to object to closing this issue during that timeframe.