Upgrade Ceramic to ComposeDB

[Jkd-eth]

User Story:

As a member of the Passport workstream,
I want to transition our existing Ceramic node to ComposeDB,
So that I can utilize its indexing capabilities and a more robust decentralized environment, ensuring alignment with our Open Data principles.

Acceptance Criteria

GIVEN a Passport user is connected,
WHEN the user modifies their passport (either by adding or removing a credential),
THEN the comprehensive passport data (incl. existing stamps etc) should be recorded in our Ceramic node utilizing ComposeDB.

Tasks:

• Ensure availability of the ComposeDB node spun up by Nick at Spin up ComposeDB node #1271
• Adjust the Passport schema in line with the EAS standard, including the EIP712 signature. This schema has already been agreed upon.
• Incorporate the new schema into the Ceramic node.
• Revise the front end to be compatible with the new schema. This process should be swift.
  • Audit the new node
  • Record to the new schema
  • Update the IAM server to issue the new format VCs for writing
  • For existing VCs of the Passport holder in our DB, transfer all their stamps to the new ComposeDB node at the start of their session in the app.
  • Update the scorer to be able to validate the new VC schema.
• Amend the documentation to indicate the availability of the GraphQL interface.

Tech Details:

Passport Schema Update(s): We need to adjust the Passport schema to accommodate the new JSON VC structure. The link within the metapointer is yet to be determined (TBD).

The agreed schema is as follows:

{
  "@context": [
    "https://www.w3.org/2018/credentials/v1",
    "https://w3id.org/vc/status-list/2021/v1"
  ],
  "type": [
    "VerifiableCredential"
  ],
  "credentialSubject": {
    "id": "did:pkh:eip155:1:0x12FeD9f987bc340c6bE43fD80aD641E8cD740682",
    "@context": {
      "hash": "https://schema.org/Text",
      "metaPointer": "https://schema.org/URL",
      "provider": "https://schema.org/Text",
      "customInfo": "https://schema.org/Thing"
    },
    "hash": "v0.0.0:AjcRjxx7Hp3PKPSNwPeBJjR21pLyA14CVeQ1XijzxUc=",
    "provider": "Twitter",
    "metaPointer": "https://github.com/gitcoinco/passport-scorer/blob/main/api/scorer/settings/gitcoin_passport_weights.py",
    "customInfo": {}
  },
  "issuer": "did:ethr:0xd6fc34345bc8c8e5659a35bed9629d5558d48c4e",
  "issuanceDate": "2022-07-19T10:42:24.883Z",
  "expiryDate": "2023-12-31T23:59:59.000Z",
  "proof": {
    "@context": "https://w3id.org/security/suites/eip712sig-2021/v1",
    "type": "EthereumEip712Signature2021",
    "proofPurpose": "assertionMethod",
    "proofValue": "0x0afc5a42d51c0f75cde083f0f55f70273eed4191cbc92d8d6f6ad7bbcefa74ca4de68618757e2a77e85239819e393f912e515f0544d23278e3e9b9d4ebc9c9301c",
    "verificationMethod": "did:ethr:0xd6fc34345bc8c8e5659a35bed9629d5558d48c4e#controller",
    "created": "2023-05-16T07:28:56.083Z"
  },
  "credentialStatus": {
    "id": "https://example.edu/credentials/status/3#94567",
    "type": "StatusList2021Entry",
    "statusPurpose": "revocation",
    "statusListIndex": "94567",
    "statusListCredential": "https://example.edu/credentials/status/3"
  }
}

Open Questions:

Notes/Assumptions:

The migration of the Snapshot SDK is currently in progress with Sophia and Zakk, and is assumed to be completed soon.

[tim-schultz]

Heres an example using didkit, to issue VCs with an EIP712 signature https://github.com/schultztimothy/passport-vc-verification/blob/main/utils/sign.ts#L22. Didkit does not provide much documentation for this signature type so wanted to share an example.

[lucianHymer]

Here are some musings I found about DID revocation: https://github.com/Open-Attestation/adr/blob/master/did-certificate-revocation.md

I don't totally love either method outlined above, I think we could do sort of a hybrid: an online authority which signs proofs that can be checked offline.

I like the idea of having
{"revocationAuthority": <address> and "revocationStatus": http://<revocation_service_base_url>/<this_did_id>}
in our VC.

The URL will point to an API that they could call to check if this did has been revoked. We could sign something like
{id: <did_id>, timestamp: <now>, revoked: <status>}
If "revoked" is false, then users could take this revocation proof and their DID and together that can still act as a decentralized, offline proof that they have a sufficiently recently non-revoked DID.

We could support bulk requests or potentially let users download the full list as well.

[nutrina]

This is a gist that issues VCs in the new format with the new signature: https://gist.github.com/nutrina/9306e82080d34abc33aa696753c6c4f2

[Jkd-eth]

Need to create additional issues for all of the work needed to complete

[nutrina]

Have updated gist to account for the latest changes (includes the customInfo now): https://gist.github.com/nutrina/9306e82080d34abc33aa696753c6c4f2

[nutrina]

I would propose the following plan to get this implemented and released:

Release step 1 - roll out the tech in the backend services

This should be pushed to prod before the actual go-live.
This sill roll out the required tech on our backend services.

• Updates to IAM: Implement New Credential Issuance Schema in IAM Server #1540
• Updates to Scorer: Update Scorer to Handle Both Stamp Formats #1541
• Updates to Scorer - remove backfilling data from Ceramic: Updates to Scorer - Remove Backfilling Data from Ceramic #1543

Release Step 2 - go live

This should be preferably be performed in a period of relatively low traffic (no new events launching that week).
This will result in a higher load in our service due to the on-the-fly conversion of stamps when reading from the Passport App. This means before rolling this out we should ensure higher compute power on our end. This should gradually go down, while old stamps will expires, and should be back to normal after 90 days.

• Support for reading & writing to / from Compose in the new schema format: Implement ComposeDatabase for Reading & Writing from Compose #1545
• Drop back-filling from ceramic: Update Passport App to Load Data Exclusively from Scorer Backend #1544
• Update the Passport App: Implementing New Stamp Format Support in the Passport App #1542

Step 3 (post go-live cleanup)

Should be performed when we are confident that it is not relevant for the user any more to have the on-the-fly ocnversion of the stamps in place (for example after 90 days all old stamps of the users should be expired, so this should be safe to do):

• Drop the on-the-fly conversion in the scorers ceramic-cache API: Post-Migration Cleanup: Removal of Redundant Processes and Code Related to Old Stamp Format passport-scorer#333

[erichfi]

Thank you so much @nutrina -- closing this issue as we've broken it down into additional tasks.