feat: add static uploads API

[alanshaw]

This PR adds static methods allowing streaming uploads to the service. It is integrated with ucanto 0.9.

Essentially the client is "bring your own agent" - it requires an issuer (a Signer) and proofs (delegated capabilities) to be passed to each method.

The API is oriented around 3 use cases:

1. Simple and easy to use uploadFile/uploadDirectory methods that return CIDs.
2. More advanced APIs that split out DAG creation, CAR creation, storing and registering uploads.
3. Streaming APIs that have the same split, but cater for uploads of arbitrary size using streams.

Every method has tests and the README has extensive examples and API reference documentation.

closes #150

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Pages

Latest commit:	7773276
Status:	✅  Deploy successful!
Preview URL:	https://ec35c83f.ucan-protocol.pages.dev
Branch Preview URL:	https://feat-static-uploading-api.ucan-protocol.pages.dev

View logs

[hugomrdias]

does this closes #150 ?

[hugomrdias]

you need to add

"references": [{ "path": "../access" }]

to tsconfig.json

[vasco-santos]

This is amazing ❤️ Learned so much

[olizilla]

can can

[olizilla]

Beautiful work ✨🎷🐩

[olizilla]

How does this work? what ties issuer + account together? why is issuer allowed to invoke an upload with this proof chain?

[Gozala]

How does this work? what ties issuer + account together?

I'm not sure I understand what you mean, here also what test does here is hopefully not what we do in the actual client. Idea is that space should delegate all of it's capabilities, that is { can: *, with: space.did() } to the service DID so that service in the future could delegate it back to the user.

Problem here is that it delegates just specific capabilities not all of them. That would imply that if user looses key we can delegate back only these two capabilities but nothing else.

why is issuer allowed to invoke an upload with this proof chain?

This is precisely what the ownership section here tries to address.

If with DID is the same as iss DID you don't need any proof because it's self-evident that you own that resource, otherwise how would you would not be able to issue such UCAN because you would not be able to sign without a private key.

P.S.: If you think there is a good / better place way to capture that let me know

[Gozala]

If you mean how does space tied to the agent then

1. On space creation:
   a. space delegates { can: *, with: space.did() } to agent.did() - So agent can act on user behalf
   b. space delegates { can: *, with: space.did() } to service.did() - So service can delegate to users other agent after email verification. Key recovery & rotation is the same flow.
   c. space should delegate to user email as per During registration we should delegate from space to an email of the user #165
2. On login to existing space
   a. User types email address and agent asks service for the delegation to (local) agent.did()
   b. Service delegates capabilities from 1. .b to requesting agent.did() and sends an email with a link to import it. After clicking it agent imports delegation and now has access to the space.

As an aside we would be able to replace delegation form space -> service, with space to user email address so we never actually have user space capabilities, because with ucan-mailto will be able to do email to did delegations

[olizilla]

my question was really "why is the issuer allowed to invoke store/add on this space when the proof chain the issuer provides doesn't include a delegation from space to issuer".

[olizilla]

this test passes and i assumed it would fail as it is currently written.

[olizilla]

that maybe just how it works, i'm just trying to clarify my concern around such a self evident proof looking like maybe it is dangerously powerful. Why is a proof chain that never addresses our signer called issuer as the audience for the capbailities usable by issuer to prove they can do a thing?

[Gozala]

i'm still not getting it! In this test we do this:

     const account = await Signer.generate()
     const issuer = await Signer.generate() // The "user" that will ask the service to accept the upload

     const proofs = await Promise.all([
       StoreCapabilities.add.delegate({
         issuer: account,
         audience: serviceSigner,
         with: account.did(),
         expiration: Infinity,
       }),
       UploadCapabilities.add.delegate({
         issuer: account,
         audience: serviceSigner,
         with: account.did(),
         expiration: Infinity,
       }),
     ])

    // ... trim server and conn creation ...

    const dataCID = await uploadFile({ issuer, proofs }, file, { connection } )

to my eyes that looks like we set up a proof that says

I don't know what uploadFile does there but assuming it issues some invocation using proofs as proofs it should not work. Issuer there should be an account or delegations should change to delegate to issuer instead of serviceSigner

[Gozala]

Again it kind of depends what uploadFile does it may be using issuer.did() in with field in which case proofs are pointless and validator won't look at them because issuer could issue any capability as long as with is issuer.did() without proofs

[Gozala]

that maybe just how it works, i'm just trying to clarify my concern around such a self evident proof looking like maybe it is dangerously powerful. Why is a proof chain that never addresses our signer called issuer as the audience for the capbailities usable by issuer to prove they can do a thing?

No if what happens under the hood looks roughly like the following, validator would have to say you're not authorized and if it does not I'm very surprised that such bug has escaped:

{ 
   iss: 'did:key:iss'
   aud: 'did:key:service'
   att: [
       { can: 'store/add', with: 'did:key:acc', link: 'bag...car' },
   ],
  proofs: [
      { iss: 'did:key:acc'
        aud: 'did:key:service'
        att: [{ can: "store/add", with: "did:key:acc" }]
      }
  ]
}

[Gozala]

I see now that what @olizilla was describing is actually what's in this test, I'm surprised it's passing. I have created test in ucanto to check for principal aligment storacha/ucanto#134

I'll continue investigating why this isn't failing