A bucket serves as a logical container for storing objects in Greenfield. Each bucket is assigned a unique name by the user at the time of creation. It's essential that bucket names be globally unique within the Greenfield namespace to avoid duplication of names. Additionally, bucket names must conform to DNS naming conventions, consisting of one or more labels separated by periods.
Objects can be uploaded to a bucket using various methods, such as the gnfd cmd or SDKs, after the bucket has been created. Objects within a bucket can be organized and managed like folders, also referred to as "prefixes". Besides, each object is assigned a unique key (a string value) to identify it within the bucket.
Each user account can create multiple buckets, with the account owning each created bucket. Every bucket should have its Primary SP associated with it, along with payment accounts for Read and Store. The owner's address will be the default payment account.
An object represents a fundamental unit of storage in Greenfield, consisting of both data and associated metadata. Each object has a unique name (a string value) within a bucket, identifying it.
Although objects are primarily used to store files in Greenfield, they can contain any data type, including text, images, videos, and program binaries. Users can upload, download, copy, or move objects using various methods such as the gnfd cmd and SDKs.
Objects in Greenfield have several critical features, including their name and ID, owner, hosting bucket, size and timestamps, content type, checkSums for storage pieces, storage status, and associated SP information. Object metadata is stored using the bucket name as its prefix, allowing iteration through all objects within the same bucket. However, doing so for a large bucket with numerous objects may require considerable effort.
Objects in Greenfield have several important characteristics, including:
- name and ID
- owner
- bucket that hosts it
- size and timestamps
- content type
- checkSums for the storage pieces
- storage status
- associated SP information
- tags
A Group is a collection of accounts that share the same permissions. The group name cannot be duplicated under the same user. However, a group cannot create or own any resources, nor can it be a member of another group.
To ensure that the on-chain permission check can be completed within a constant time, only a limited number of groups can be associated with a resource for permissions.
The storage module keeps state of the following primary objects:
- BucketInfo
- ObjectInfo:
- GroupInfo
The primary objects are intended to be stored and accessed mainly using the auto-incremented sequence ID.
However, additional indices are also maintained for each primary object to ensure compatibility with the S3 object storage.
-
BucketInfo:
0x11 | hash(bucketName) -> BigEndian(bucketId) -
ObjectInfo:
0x12 | hash(bucketName)_hash(objectName) -> BigEndian(objectId) -
GroupInfo:
0x13 | OwnerAddr_hash(groupName) -> BigEndian(groupId) -
BucketInfoById:
0x21 | BigEndian(bucketId) -> ProtoBuf(BucketInfo) -
ObjectInfoById:
0x22 | BigEndian(objectId) -> ProtoBuf(ObjectInfo) -
GroupInfoById:
0x23 | BigEndian(groupId) -> ProtoBuf(GroupInfo)
The storage module contains the following parameters, they can be updated with governance.
| name | default value | meaning |
|---|---|---|
| MaxSegmentSize | 16M | The maximum size of the segment. The payload data of an object will split into several segment. Only the size of the last segment can be less than MaxSegmentSize, others is equals. |
| RedundantDataChunkNum | 4 | The number of the data chunks in Erasure-Code algorithm. |
| RedundantParityChunkNum | 2 | The number of the parity chunks in Erasure-Code algorithm. |
| MaxPayloadSize | 32G | The maximum size of the payload data that allowed in greenfield storage network. |
| MinChargeSize | 128KB | The minimum charge size of the payload, objects smaller than this size will be charged as this size |
| MaxBucketsPerAccount | 100 | The maximum number of buckets that can be created per account |
In this section, we'll list out the various messages that are associated with storage modules.
Used to create a new bucket. Buckets are used to contain storage objects.
message MsgCreateBucket {
option (cosmos.msg.v1.signer) = "creator";
// creator defines the account address of bucket creator, it is also the bucket owner.
string creator = 1 [(cosmos_proto.scalar) = "cosmos.AddressString"];
// bucket_name defines a globally unique name of bucket
string bucket_name = 2;
// visibility means the bucket is private or public. if private, only bucket owner or grantee can read it,
// otherwise every greenfield user can read it.
VisibilityType visibility = 3;
// payment_address defines an account address specified by bucket owner to pay the read fee. Default: creator
string payment_address = 4 [(cosmos_proto.scalar) = "cosmos.AddressString"];
// primary_sp_address defines the address of primary sp.
string primary_sp_address = 5 [(cosmos_proto.scalar) = "cosmos.AddressString"];
// primary_sp_approval defines the approval info of the primary SP which indicates that primary sp confirm the user's request.
common.Approval primary_sp_approval = 6;
// charged_read_quota defines the read data that users are charged for, measured in bytes.
// The available read data for each user is the sum of the free read data provided by SP and
// the ChargeReadQuota specified here.
uint64 charged_read_quota = 7;
}Used to delete an existing bucket. It is important to note that you cannot delete a non-empty bucket.
message MsgDeleteBucket {
option (cosmos.msg.v1.signer) = "operator";
// creator defines the account address of the grantee who has the DeleteBucket permission of the bucket to be deleted.
string operator = 1 [(cosmos_proto.scalar) = "cosmos.AddressString"];
// bucket_name defines the name of the bucket to be deleted.
string bucket_name = 2;
}Used to update the information in a bucket.
message MsgUpdateBucketInfo {
option (cosmos.msg.v1.signer) = "operator";
// operator defines the account address of the operator
string operator = 1 [(cosmos_proto.scalar) = "cosmos.AddressString"];
// bucket_name defines the name of bucket which you'll update
string bucket_name = 2;
// charged_read_quota defines the traffic quota that you read from primary sp
// if read_quota is nil, it means don't change the read_quota
common.UInt64Value charged_read_quota = 3;
// payment_address defines the account address of the payment account
// if payment_address is empty, it means don't change the payment_address
string payment_address = 4 [(cosmos_proto.scalar) = "cosmos.AddressString"];
// visibility means the bucket is private or public. if private, only bucket owner or grantee can read it,
// otherwise every greenfield user can read it.
VisibilityType visibility = 5;
}Used to create an initial object under a bucket.
message MsgCreateObject {
option (cosmos.msg.v1.signer) = "creator";
// creator defines the account address of object uploader
string creator = 1 [(cosmos_proto.scalar) = "cosmos.AddressString"];
// bucket_name defines the name of the bucket where the object is stored.
string bucket_name = 2;
// object_name defines the name of object
string object_name = 3;
// payload_size defines size of the object's payload
uint64 payload_size = 4;
// visibility means the object is private or public. if private, only object owner or grantee can access it,
// otherwise every greenfield user can access it.
VisibilityType visibility = 5;
// content_type defines a standard MIME type describing the format of the object.
string content_type = 6;
// primary_sp_approval defines the approval info of the primary SP which indicates that primary sp confirm the user's request.
Approval primary_sp_approval = 7;
// expect_checksums defines a list of hashes which was generate by redundancy algorithm.
repeated bytes expect_checksums = 8;
// redundancy_type can be ec or replica
RedundancyType redundancy_type = 9;
}Used to delete object that is no longer useful, the deleted storage object is not recoverable.
message MsgDeleteObject {
option (cosmos.msg.v1.signer) = "operator";
// operator defines the account address of the operator who has the DeleteObject permission of the object to be deleted.
string operator = 1 [(cosmos_proto.scalar) = "cosmos.AddressString"];
// bucket_name defines the name of the bucket where the object which to be deleted is stored.
string bucket_name = 2;
// object_name defines the name of the object which to be deleted.
string object_name = 3;
}
With this message, Storage providers seal an object once the underlying files are well saved by both primary and second SPs.
message MsgSealObject {
option (cosmos.msg.v1.signer) = "operator";
// operator defines the account address of primary SP
string operator = 1 [(cosmos_proto.scalar) = "cosmos.AddressString"];
// bucket_name defines the name of the bucket where the object is stored.
string bucket_name = 2;
// object_name defines the name of object to be sealed.
string object_name = 3;
// global_virtual_group_id defines the id of global virtual group
uint32 global_virtual_group_id = 4;
// secondary_sp_bls_agg_signatures defines the aggregate bls signature of the secondary sp that can
// acknowledge that the payload data has received and stored.
bytes secondary_sp_bls_agg_signatures = 5;
}Used to send a copy of an object to another user.
message MsgCopyObject {
option (cosmos.msg.v1.signer) = "operator";
// operator defines the account address of the operator who has the CopyObject permission.
string operator = 1;
// src_bucket_name defines the name of the bucket where the object to be copied is located
string src_bucket_name = 2;
// dst_bucket_name defines the name of the bucket where the object is copied to.
string dst_bucket_name = 3;
// src_object_name defines the name of the object which to be copied
string src_object_name = 4;
// dst_object_name defines the name of the object which is copied to
string dst_object_name = 5;
// primary_sp_approval defines the approval info of the primary SP which indicates that primary sp confirm the user's request.
common.Approval dst_primary_sp_approval = 6;
}A storage provider may reject to seal an object if it refuses to, or be unable to do so due to unexpected errors.
message MsgRejectSealObject {
option (cosmos.msg.v1.signer) = "operator";
// operator defines the account address of the object owner
string operator = 1 [(cosmos_proto.scalar) = "cosmos.AddressString"];
// bucket_name defines the name of the bucket where the object is stored.
string bucket_name = 2;
// object_name defines the name of unsealed object to be reject.
string object_name = 3;
}The users are able to cancel an initial object before it is sealed with this message.
message MsgCancelCreateObject {
option (cosmos.msg.v1.signer) = "operator";
// operator defines the account address of the operator
string operator = 1 [(cosmos_proto.scalar) = "cosmos.AddressString"];
// bucket_name defines the name of the bucket
string bucket_name = 2;
// object_name defines the name of the object
string object_name = 3;
}Used to create a new group.
message MsgCreateGroup {
option (cosmos.msg.v1.signer) = "creator";
// owner defines the account address of group owner who create the group
string creator = 1 [(cosmos_proto.scalar) = "cosmos.AddressString"];
// group_name defines the name of the group. it's not globally unique.
string group_name = 2;
// extra defines extra info for the group
string extra = 3;
}Used to delete a group that is no longer used. Please note that the underlying members are not deleted yet.
message MsgDeleteGroup {
option (cosmos.msg.v1.signer) = "operator";
// operator defines the account address of the operator who has the DeleteGroup permission of the group to be deleted.
string operator = 1 [(cosmos_proto.scalar) = "cosmos.AddressString"];
// group_name defines the name of the group which to be deleted
string group_name = 2;
}
A group member can choose to leave a group by sending this message.
message MsgLeaveGroup {
option (cosmos.msg.v1.signer) = "member";
// member defines the account address of the member who want to leave the group
string member = 1 [(cosmos_proto.scalar) = "cosmos.AddressString"];
// group_owner defines the owner of the group you want to leave
string group_owner = 2 [(cosmos_proto.scalar) = "cosmos.AddressString"];
// group_name defines the name of the group you want to leave
string group_name = 3;
}Mirror an object to the destination chain as NFT.
message MsgMirrorObject {
option (cosmos.msg.v1.signer) = "operator";
// operator defines the account address of the object owner.
string operator = 1 [(cosmos_proto.scalar) = "cosmos.AddressString"];
// id defines the unique u256 for object.
string id = 2 [
(cosmos_proto.scalar) = "cosmos.Uint",
(gogoproto.customtype) = "Uint",
(gogoproto.nullable) = false
];
// bucket_name defines the name of the bucket where the object is stored
string bucket_name = 3;
// object_name defines the name of object
string object_name = 4;
// destination chain id
uint32 dest_chain_id = 5;
}Mirror a bucket to the destination chain as NFT.
message MsgMirrorBucket {
option (cosmos.msg.v1.signer) = "operator";
// operator defines the account address of the bucket owner.
string operator = 1 [(cosmos_proto.scalar) = "cosmos.AddressString"];
// id defines the unique u256 for bucket.
string id = 2 [
(cosmos_proto.scalar) = "cosmos.Uint",
(gogoproto.customtype) = "Uint",
(gogoproto.nullable) = false
];
// bucket_name defines a globally unique name of bucket
string bucket_name = 3;
// destination chain id
uint32 dest_chain_id = 4;
}Mirror a group to the destination chain as NFT.
message MsgMirrorGroup {
option (cosmos.msg.v1.signer) = "operator";
// operator defines the account address of the group owner.
string operator = 1 [(cosmos_proto.scalar) = "cosmos.AddressString"];
// id defines the unique u256 for group.
string id = 2 [
(cosmos_proto.scalar) = "cosmos.Uint",
(gogoproto.customtype) = "Uint",
(gogoproto.nullable) = false
];
// group_name defines the name of the group
string group_name = 3;
// destination chain id
uint32 dest_chain_id = 4;
}Migrate a bucket to another primary SP.
message MsgMigrateBucket {
option (cosmos.msg.v1.signer) = "operator";
// operator defines the account address of the operator who initial the migrate bucket
string operator = 1 [(cosmos_proto.scalar) = "cosmos.AddressString"];
// bucket_name defines the name of the bucket that need to be migrated
string bucket_name = 2;
// dst_primary_sp_id defines the destination SP for migration
uint32 dst_primary_sp_id = 3;
// dst_primary_sp_approval defines the approval of destination sp
common.Approval dst_primary_sp_approval = 4;
}Cancel an existing bucket migration.
message MsgCancelMigrateBucket {
option (cosmos.msg.v1.signer) = "operator";
// operator defines the account address of the msg operator.
// Only the user can send this transaction to cancel the migrate bucket
string operator = 1 [(cosmos_proto.scalar) = "cosmos.AddressString"];
// bucket_name defines the name of the bucket that need to be migrated
string bucket_name = 2;
}Used to update the tags of a resource. The old tags will be overwritten directly.
message MsgSetTag {
option (cosmos.msg.v1.signer) = "operator";
// operator defines the operator who adds the tags
string operator = 1 [(cosmos_proto.scalar) = "cosmos.AddressString"];
// resource defines a greenfield standard resource name that can be generated by GRN structure
string resource = 2;
// tags defines a list of tags which will be set to the resource
ResourceTags tags = 3;
}Used to update an existing object content
message MsgUpdateObjectContent {
option (cosmos.msg.v1.signer) = "operator";
// operator defines the account address of the operator, either the object owner or the updater with granted permission.
string operator = 1 [(cosmos_proto.scalar) = "cosmos.AddressString"];
// bucket_name defines the name of the bucket where the object is stored.
string bucket_name = 2;
// object_name defines the name of object
string object_name = 3;
// payload_size defines size of the object's payload
uint64 payload_size = 4;
// content_type defines a standard MIME type describing the format of the object.
string content_type = 5;
// expect_checksums defines a list of hashes which was generate by redundancy algorithm.
repeated bytes expect_checksums = 6;
}Used to cancel the update on an existing object
message MsgCancelUpdateObjectContent {
option (cosmos.msg.v1.signer) = "operator";
// operator defines the account address of the operator, either the object owner or the updater with granted permission.
string operator = 1 [(cosmos_proto.scalar) = "cosmos.AddressString"];
// bucket_name defines the name of the bucket
string bucket_name = 2;
// object_name defines the name of the object
string object_name = 3;
}