Skip to content

Commit

Permalink
Fix other parts
Browse files Browse the repository at this point in the history
Signed-off-by: Xuanwo <github@xuanwo.io>
  • Loading branch information
Xuanwo committed Jul 11, 2022
1 parent 4140482 commit a4d2b54
Showing 1 changed file with 77 additions and 9 deletions.
86 changes: 77 additions & 9 deletions docs/rfcs/0438-multipart.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
- Proposal Name: `multipart`
- Start Date: 2022-07-11
- RFC PR: [datafuselabs/opendal#438](https://github.com/datafuselabs/opendal/pull/438)
- Tracking Issue: [datafuselabs/opendal#0000](https://github.com/datafuselabs/opendal/issues/0000)
- Tracking Issue: [datafuselabs/opendal#439](https://github.com/datafuselabs/opendal/issues/439)

# Summary

Expand All @@ -14,7 +14,7 @@ Add multipart support in OpenDAL.
A successful multipart upload includes the following steps:

- `CreateMultipartUpload`: Start a new multipart upload.
- `UploadPart`: Upload a single part with the previously got upload id.
- `UploadPart`: Upload a single part with the previously uploaded id.
- `CompleteMultipartUpload`: Complete a multipart upload to get a regular object.

To cancel a multipart upload, users need to call `AbortMultipartUpload`.
Expand Down Expand Up @@ -52,7 +52,7 @@ With `Multipart`, we can upload a new part:
let part = mp.write(part_number, content).await?;
```

After all parts has been uploaded, we can finish this upload:
After all parts have been uploaded, we can finish this upload:

```rust
let _ = mp.complete(parts).await?;
Expand All @@ -66,19 +66,87 @@ let _ = mp.abort().await?;

# Reference-level explanation

TBD
`Accessor` will add the following APIs:

```rust
pub trait Accessor: Send + Sync + Debug {
async fn create_multipart(&self, args: &OpCreateMultipart) -> Result<String> {
let _ = args;
unimplemented!()
}

async fn write_multipart(&self, args: &OpWriteMultipart) -> Result<PartWriter> {
let _ = args;
unimplemented!()
}

async fn complete_multipart(&self, args: &OpCompleteMultipart) -> Result<()> {
let _ = args;
unimplemented!()
}

async fn abort_multipart(&self, args: &OpAbortMultipart) -> Result<()> {
let _ = args;
unimplemented!()
}
}
```

While closing a `PartWriter`, a `Part` will be generated.

`Operator` will build APIs based on `Accessor`:

```rust
impl Object {
async fn create_multipart(&self) -> Result<Multipart> {}
fn into_multipart(&self, upload_id: &str) -> Multipart {}
}

impl Multipart {
async fn write(&self, part_number: usize, bs: impl AsRef<[u8]>) -> Result<Part> {}
async fn writer(&self, part_number: usize, size: u64) -> Result<impl PartWrite> {}
async fn complete(&self, ps: &[Part]) -> Result<()> {}
async fn abort(&self) -> Result<()> {}
}
```

# Drawbacks

TBD
None.

# Rationale and alternatives

TBD
## Why not add new object modes?

It seems natural to add a new object mode like `multipart`.

```rust
pub enum ObjectMode {
FILE,
DIR,
MULTIPART,
Unknown,
}
```

However, to make this work, we need big API breaks that introduce `mode` in Object.

And we need to change every API call to accept `mode` as args.

For example:

```rust
let _ = op.object("path/to/dir/").list(ObjectMODE::MULTIPART);
let _ = op.object("path/to/file").stat(ObjectMODE::MULTIPART)
```

## Why not split Object into File and Dir?

We can split `Object` into `File` and `Dir` to avoid requiring `mode` in API. There is a vast API breakage too.

# Prior art

TBD
None.

# Unresolved questions

Expand All @@ -88,8 +156,8 @@ None.

## Support list multipart uploads

We can support list multipart uploads to list ongoing multipart uploads. So that we can resume an upload or abort them.
We can support listing multipart uploads to list ongoing multipart uploads so we can resume an upload or abort them.

## Support list part

We can support list parts to list already uploaded parts for an upload.
We can support listing parts to list already uploaded parts for an upload.

1 comment on commit a4d2b54

@github-actions
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deploy preview for opendal ready!

✅ Preview
https://opendal-tsn8x8pkv-databend.vercel.app

Built with commit a4d2b54.
This pull request is being automatically deployed with vercel-action

Please sign in to comment.