Skip to content

Commit

Permalink
feat: explicit resource (#181)
Browse files Browse the repository at this point in the history
  • Loading branch information
Alan Shaw authored Nov 18, 2022
1 parent b0d1c95 commit 785924b
Show file tree
Hide file tree
Showing 13 changed files with 354 additions and 357 deletions.
22 changes: 14 additions & 8 deletions packages/upload-client/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,27 +15,33 @@ npm install @web3-storage/upload-client

### Create an Agent

An Agent provides an `issuer` (a key linked to your account) and `proofs` to show your `issuer` has been delegated the capabilities to store data and register uploads.
An Agent provides:

1. The key pair used to call the service and sign the payload (the `issuer`).
2. A decentralized identifier (DID) of the "space" where data should be uploaded (the `with`).
3. Proof showing your `issuer` has been delegated capabilities to store data and register uploads to the "space" (`proofs`).

```js
import { Agent } from '@web3-storage/access'
import { add as storeAdd } from '@web3-storage/access/capabilities/store'
import { add as uploadAdd } from '@web3-storage/access/capabilities/upload'
import { store } from '@web3-storage/access/capabilities/store'
import { upload } from '@web3-storage/access/capabilities/upload'

const agent = await Agent.create({ store })
const agent = await Agent.create()

// Note: you need to create and register an account 1st time:
// await agent.createAccount('you@youremail.com')
// Note: you need to create and register a space 1st time:
// await agent.createSpace()
// await agent.registerSpace('you@youremail.com')

const conf = {
issuer: agent.issuer,
proofs: agent.getProofs([storeAdd, uploadAdd]),
with: agent.currentSpace(),
proofs: agent.getProofs([store, upload]),
}
```

### Uploading files

Once you have the `issuer` and `proofs`, you can upload a directory of files by passing that invocation config to `uploadDirectory` along with your list of files to upload.
Once you have the `issuer`, `with` and `proofs`, you can upload a directory of files by passing that invocation config to `uploadDirectory` along with your list of files to upload.

You can get your list of Files from a [`<input type="file">`](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input/file) element in the browser or using [`files-from-path`](https://npm.im/files-from-path) in Node.js

Expand Down
30 changes: 18 additions & 12 deletions packages/upload-client/src/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -18,22 +18,25 @@ export * from './sharding.js'
*
* Required delegated capability proofs: `store/add`, `upload/add`
*
* @param {import('./types').InvocationConfig} invocationConfig Configuration
* for the UCAN invocation. An object with `issuer` and `proofs`.
* @param {import('./types').InvocationConfig} conf Configuration
* for the UCAN invocation. An object with `issuer`, `with` and `proofs`.
*
* The `issuer` is the signing authority that is issuing the UCAN
* invocation(s). It is typically the user _agent_.
*
* The `with` is the resource the invocation applies to. It is typically the
* DID of a space.
*
* The `proofs` are a set of capability delegations that prove the issuer
* has the capability to perform the action.
*
* The issuer needs the `store/add` and `upload/add` delegated capability.
* @param {Blob} file File data.
* @param {UploadOptions} [options]
*/
export async function uploadFile({ issuer, proofs }, file, options = {}) {
export async function uploadFile(conf, file, options = {}) {
return await uploadBlockStream(
{ issuer, proofs },
conf,
UnixFS.createFileEncoderStream(file),
options
)
Expand All @@ -46,41 +49,44 @@ export async function uploadFile({ issuer, proofs }, file, options = {}) {
*
* Required delegated capability proofs: `store/add`, `upload/add`
*
* @param {import('./types').InvocationConfig} invocationConfig Configuration
* for the UCAN invocation. An object with `issuer` and `proofs`.
* @param {import('./types').InvocationConfig} conf Configuration
* for the UCAN invocation. An object with `issuer`, `with` and `proofs`.
*
* The `issuer` is the signing authority that is issuing the UCAN
* invocation(s). It is typically the user _agent_.
*
* The `with` is the resource the invocation applies to. It is typically the
* DID of a space.
*
* The `proofs` are a set of capability delegations that prove the issuer
* has the capability to perform the action.
*
* The issuer needs the `store/add` and `upload/add` delegated capability.
* @param {import('./types').FileLike[]} files File data.
* @param {UploadOptions} [options]
*/
export async function uploadDirectory({ issuer, proofs }, files, options = {}) {
export async function uploadDirectory(conf, files, options = {}) {
return await uploadBlockStream(
{ issuer, proofs },
conf,
UnixFS.createDirectoryEncoderStream(files),
options
)
}

/**
* @param {import('./types').InvocationConfig} invocationConfig
* @param {import('./types').InvocationConfig} conf
* @param {ReadableStream<import('@ipld/unixfs').Block>} blocks
* @param {UploadOptions} [options]
* @returns {Promise<import('./types').AnyLink>}
*/
async function uploadBlockStream({ issuer, proofs }, blocks, options = {}) {
async function uploadBlockStream(conf, blocks, options = {}) {
/** @type {import('./types').CARLink[]} */
const shards = []
/** @type {import('./types').AnyLink?} */
let root = null
await blocks
.pipeThrough(new ShardingStream())
.pipeThrough(new ShardStoringStream({ issuer, proofs }, options))
.pipeThrough(new ShardStoringStream(conf, options))
.pipeTo(
new WritableStream({
write(meta) {
Expand All @@ -94,6 +100,6 @@ async function uploadBlockStream({ issuer, proofs }, blocks, options = {}) {
/* c8 ignore next */
if (!root) throw new Error('missing root CID')

await Upload.add({ issuer, proofs }, root, shards, options)
await Upload.add(conf, root, shards, options)
return root
}
4 changes: 2 additions & 2 deletions packages/upload-client/src/service.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,13 @@ import * as DID from '@ipld/dag-ucan/did'
export const serviceURL = new URL(
'https://8609r1772a.execute-api.us-east-1.amazonaws.com'
)
export const serviceDID = DID.parse(
export const servicePrincipal = DID.parse(
'did:key:z6MkrZ1r5XBFZjBU34qyD8fueMbMRkKw17BZaq2ivKFjnz2z'
)

/** @type {import('@ucanto/interface').ConnectionView<import('./types').Service>} */
export const connection = connect({
id: serviceDID,
id: servicePrincipal,
encoder: CAR,
decoder: CBOR,
channel: HTTP.open({
Expand Down
11 changes: 7 additions & 4 deletions packages/upload-client/src/sharding.js
Original file line number Diff line number Diff line change
Expand Up @@ -67,19 +67,22 @@ export class ShardingStream extends TransformStream {
*/
export class ShardStoringStream extends TransformStream {
/**
* @param {import('./types').InvocationConfig} invocationConfig Configuration
* for the UCAN invocation. An object with `issuer` and `proofs`.
* @param {import('./types').InvocationConfig} conf Configuration
* for the UCAN invocation. An object with `issuer`, `with` and `proofs`.
*
* The `issuer` is the signing authority that is issuing the UCAN
* invocation(s). It is typically the user _agent_.
*
* The `with` is the resource the invocation applies to. It is typically the
* DID of a space.
*
* The `proofs` are a set of capability delegations that prove the issuer
* has the capability to perform the action.
*
* The issuer needs the `store/add` delegated capability.
* @param {import('./types').RequestOptions} [options]
*/
constructor({ issuer, proofs }, options = {}) {
constructor(conf, options = {}) {
const queue = new Queue({ concurrency: CONCURRENT_UPLOADS })
const abortController = new AbortController()
super({
Expand All @@ -88,7 +91,7 @@ export class ShardStoringStream extends TransformStream {
async () => {
try {
const opts = { ...options, signal: abortController.signal }
const cid = await add({ issuer, proofs }, car, opts)
const cid = await add(conf, car, opts)
const { version, roots, size } = car
controller.enqueue({ version, roots, cid, size })
} catch (err) {
Expand Down
63 changes: 38 additions & 25 deletions packages/upload-client/src/store.js
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
import { CAR } from '@ucanto/transport'
import * as StoreCapabilities from '@web3-storage/access/capabilities/store'
import retry, { AbortError } from 'p-retry'
import { serviceDID, connection } from './service.js'
import { findCapability } from './utils.js'
import { servicePrincipal, connection } from './service.js'
import { REQUEST_RETRIES } from './constants.js'

/**
Expand All @@ -11,12 +10,15 @@ import { REQUEST_RETRIES } from './constants.js'
*
* Required delegated capability proofs: `store/add`
*
* @param {import('./types').InvocationConfig} invocationConfig Configuration
* for the UCAN invocation. An object with `issuer` and `proofs`.
* @param {import('./types').InvocationConfig} conf Configuration
* for the UCAN invocation. An object with `issuer`, `with` and `proofs`.
*
* The `issuer` is the signing authority that is issuing the UCAN
* invocation(s). It is typically the user _agent_.
*
* The `with` is the resource the invocation applies to. It is typically the
* DID of a space.
*
* The `proofs` are a set of capability delegations that prove the issuer
* has the capability to perform the action.
*
Expand All @@ -25,8 +27,11 @@ import { REQUEST_RETRIES } from './constants.js'
* @param {import('./types').RequestOptions} [options]
* @returns {Promise<import('./types').CARLink>}
*/
export async function add({ issuer, proofs }, car, options = {}) {
const capability = findCapability(proofs, StoreCapabilities.add.can)
export async function add(
{ issuer, with: resource, proofs, audience = servicePrincipal },
car,
options = {}
) {
// TODO: validate blob contains CAR data
const bytes = new Uint8Array(await car.arrayBuffer())
const link = await CAR.codec.link(bytes)
Expand All @@ -37,9 +42,8 @@ export async function add({ issuer, proofs }, car, options = {}) {
return await StoreCapabilities.add
.invoke({
issuer,
audience: serviceDID,
// @ts-expect-error expects did:${string} but cap with is ${string}:${string}
with: capability.with,
audience,
with: resource,
nb: { link, size: car.size },
proofs,
})
Expand Down Expand Up @@ -99,29 +103,33 @@ export async function add({ issuer, proofs }, car, options = {}) {
/**
* List CAR files stored by the issuer.
*
* @param {import('./types').InvocationConfig} invocationConfig Configuration
* for the UCAN invocation. An object with `issuer` and `proofs`.
* @param {import('./types').InvocationConfig} conf Configuration
* for the UCAN invocation. An object with `issuer`, `with` and `proofs`.
*
* The `issuer` is the signing authority that is issuing the UCAN
* invocation(s). It is typically the user _agent_.
*
* The `with` is the resource the invocation applies to. It is typically the
* DID of a space.
*
* The `proofs` are a set of capability delegations that prove the issuer
* has the capability to perform the action.
*
* The issuer needs the `store/list` delegated capability.
* @param {import('./types').RequestOptions} [options]
*/
export async function list({ issuer, proofs }, options = {}) {
const capability = findCapability(proofs, StoreCapabilities.list.can)
export async function list(
{ issuer, with: resource, proofs, audience = servicePrincipal },
options = {}
) {
/* c8 ignore next */
const conn = options.connection ?? connection

const result = await StoreCapabilities.list
.invoke({
issuer,
audience: serviceDID,
// @ts-expect-error expects did:${string} but cap with is ${string}:${string}
with: capability.with,
audience,
with: resource,
proofs,
})
.execute(conn)

Expand All @@ -137,31 +145,36 @@ export async function list({ issuer, proofs }, options = {}) {
/**
* Remove a stored CAR file by CAR CID.
*
* @param {import('./types').InvocationConfig} invocationConfig Configuration
* for the UCAN invocation. An object with `issuer` and `proofs`.
* @param {import('./types').InvocationConfig} conf Configuration
* for the UCAN invocation. An object with `issuer`, `with` and `proofs`.
*
* The `issuer` is the signing authority that is issuing the UCAN
* invocation(s). It is typically the user _agent_.
*
* The `with` is the resource the invocation applies to. It is typically the
* DID of a space.
*
* The `proofs` are a set of capability delegations that prove the issuer
* has the capability to perform the action.
*
* The issuer needs the `store/remove` delegated capability.
* @param {import('./types').CARLink} link CID of CAR file to remove.
* @param {import('./types').RequestOptions} [options]
*/
export async function remove({ issuer, proofs }, link, options = {}) {
const capability = findCapability(proofs, StoreCapabilities.remove.can)
export async function remove(
{ issuer, with: resource, proofs, audience = servicePrincipal },
link,
options = {}
) {
/* c8 ignore next */
const conn = options.connection ?? connection

const result = await StoreCapabilities.remove
.invoke({
issuer,
audience: serviceDID,
// @ts-expect-error expects did:${string} but cap with is ${string}:${string}
with: capability.with,
audience,
with: resource,
nb: { link },
proofs,
})
.execute(conn)

Expand Down
19 changes: 17 additions & 2 deletions packages/upload-client/src/types.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,14 @@
import { Link, UnknownLink, Version } from 'multiformats/link'
import { Block } from '@ipld/unixfs'
import { CAR } from '@ucanto/transport'
import { ServiceMethod, ConnectionView, Signer, Proof } from '@ucanto/interface'
import {
ServiceMethod,
ConnectionView,
Signer,
Proof,
DID,
Principal,
} from '@ucanto/interface'
import {
StoreAdd,
StoreList,
Expand Down Expand Up @@ -60,9 +67,17 @@ export interface UploadListResult {

export interface InvocationConfig {
/**
* Signing authority that is issuing the UCAN invocations.
* Signing authority that is issuing the UCAN invocation(s).
*/
issuer: Signer
/**
* The principal delegated to in the current UCAN.
*/
audience?: Principal
/**
* The resource the invocation applies to.
*/
with: DID
/**
* Proof(s) the issuer has the capability to perform the action.
*/
Expand Down
Loading

0 comments on commit 785924b

Please sign in to comment.