Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs(RFC): Object Versioning #2602

Merged
merged 8 commits into from
Jul 9, 2023
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
102 changes: 102 additions & 0 deletions core/src/docs/rfcs/2602_object_versioning.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
Proposal Name: object versioning
suyanhanx marked this conversation as resolved.
Show resolved Hide resolved
Start Date: 2023-07-06
RFC PR: https://github.com/apache/incubator-opendal/pull/2602
Tracking Issue: (leave this empty)

# Summary

This proposal describes the object versioning (or object version control) feature of OpenDAL.

# Motivation

Object versioning is a common feature in many storage services.
suyanhanx marked this conversation as resolved.
Show resolved Hide resolved

# Guide-level explanation

What is object versioning?

Object versioning is a feature that allows users to keep multiple versions of an object in the same bucket.

It's a way to preserve, retrieve, and restore every version of every object stored in a bucket.

With object versioning, users can easily recover from both unintended user actions and application failures.

When object versioning is enabled, each object will have a history of versions. Each version will have a unique version ID, which is a string that is unique for each version of an object.

The version ID is not a timestamp. It is not guaranteed to be sequential.

When object versioning is enabled, the following operations will be supported:

- `stat`: Get the metadata of an object with specific version ID.
- `read`: Read a specific version of an object.
- `delete`: Delete a specific version of an object.

Code example:

```rust
// stat with version ID
let meta = op.stat_with("path/to/file").version("version_id").await?;
Xuanwo marked this conversation as resolved.
Show resolved Hide resolved
// read with version ID
let content = op.read_with("path/to/file").version("version_id").await?;
// delete with version ID
op.delete_with("path/to/file").version("version_id").await?;
```

With object versioning, users can:

- Track the history of an object.
- Implement optimistic concurrency control.
- Implement a simple backup system.

# Reference-level explanation

Those operations with object version are different from the normal operations:

- `stat`: when getting the metadata of an object, it will always get the metadata of the latest version of the object if no version ID is specified.
suyanhanx marked this conversation as resolved.
Show resolved Hide resolved
- `read`: when reading an object, it will always read the latest version of the object if no version ID is specified.
- `delete`: when deleting an object, it will always delete the latest version of the object if no version ID is specified. And users will not be able to read this object unless they specify the version ID not to be deleted.
suyanhanx marked this conversation as resolved.
Show resolved Hide resolved

And with object versioning, when writing an object,
it will always create a new version of the object than overwrite the old version.
But here it is imperceptible to the user.
Because the version id is generated by the service itself, it cannot be specified by the user and user cannot override the historical version.

To implement object versioning, we will do the following:

- Add a new field `version` to `OpStat`, `OpRead` and `OpDelete` struct.
- Add a new field `version` to `ObjectMetadata` struct.
- Add a new property(setter) `version` to the return value of `stat_with`, `read_with` method.
- Add a new method `delete_with` and add a new property(setter) `version` to the return value of `delete_with` method.

For service backend, it should support the following operations:

- `stat`: Get the metadata of an object with specific version ID.
- `read`: Read a specific version of an object.
- `delete`: Delete a specific version of an object.

## reference:
suyanhanx marked this conversation as resolved.
Show resolved Hide resolved

- [AWS S3 Object Versioning](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Versioning.html)
suyanhanx marked this conversation as resolved.
Show resolved Hide resolved
- [Google Cloud Storage Object Versioning](https://cloud.google.com/storage/docs/object-versioning)
- [Azure Blob Storage Object Versioning](https://docs.microsoft.com/en-us/azure/storage/blobs/versioning-overview)

# Drawbacks

None.

# Rationale and alternatives

None.

# Prior art
suyanhanx marked this conversation as resolved.
Show resolved Hide resolved

None.

# Unresolved questions

None.

# Future possibilities

Impl a new method `list_versions`(list all versions of an object).

3 changes: 3 additions & 0 deletions core/src/docs/rfcs/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -142,3 +142,6 @@ pub mod rfc_2133_append_api {}

#[doc = include_str!("2299_chain_based_operator_api.md")]
pub mod rfc_2299_chain_based_operator_api {}

#[doc = include_str!("2602_object_versioning.md")]
pub mod rfc_2602_object_versioning_api {}
suyanhanx marked this conversation as resolved.
Show resolved Hide resolved