Service for storing data which is traded on AMO Blockchain
- Support REST API for operations related to CRUD of data
- Support authentication based on client's identity in AMO blockchain
- Support authorization through communication with AMO blockchain
- Docker (higher than version 17)
- Docker Compose
Make your own config directory and put config.ini
and key.json
into the directory.
Example of config.ini and key.json file is in the ./config
directory.
You must set REDIS_HOST
as redis
like below.
; config.ini example
[AmoStorageConfig]
; App configurations
DEBUG=0
; SQLite Configurations
SQLALCHEMY_TRACK_MODIFICATIONS=0
SQLALCHEMY_DATABASE_URI=sqlite:////tmp/amo_storage.db
; Redis Configurations
REDIS_HOST=redis
REDIS_PORT=6379
REDIS_DB=0
; Service ID for data parcel
SERVICE_ID=00000001
[AuthConfig]
ISSUER=amo-storage
ALGORITHM=HS256
SECRET=your-sercret
[CephConfig]
HOST=127.0.0.1
PORT=7480
BUCKET_NAME=amo
[AmoBlockchainNodeConfig]
HOST=127.0.0.1
PORT=26657
NOTE: This section is not for the CEPH's configuration or configuring the CEPH cluster itself but for connecting to existing CEPH properly. It is assumed that the CEPH cluster is constructed and configured separately and it is assumed that there exists a CEPH Rados Gateway running.
- The
access_key
andsecret_key
which are used for connecting to the Rados Gateway can be found in thekey
attribute of CEPH user. (We assume there is a Rados Gateway useramoapi
for this adapter software. This may become configurable in the future version of this software.) - The
access_key
andsecret_key
should be included in thekey.json
file.
{
"access_key":"{USER'S_ACCESS_KEY}",
"secret_key":"{USER'S_SECRET_KEY}"
}
Set CONFIG_DIR
as your own config directory and set PORT
as storage service going to listen.
$ export CONFIG_DIR=/tmp
$ export PORT=5000
Or you can create .env
at the same path of docker-compose.yml
// .env example
CONFIG_DIR=/tmp/config.ini
PORT=5000
Run below command at the same path of docker-compose.yml
$ docker-compose up -d
- Python3 (compatible with 3.6.8)
- Python3-pip (compatible with 9.0.1)
- Redis (compatible with 5.0.5)
- Sqlite (compatible with 5.0.5)
Config directory should contain config.ini
and key.json
. The default configurations directory is provided with the name config
but you can specify the config directory path manually.
Most of the configurations can be set in config.ini
file except access key and secret used for connecting to CEPH.
Below is an example of config.ini
You should set configurations for your environment.
[AmoStorageConfig]
; App configurations
DEBUG=0
; SQLite Configurations
SQLALCHEMY_TRACK_MODIFICATIONS=0
SQLALCHEMY_DATABASE_URI=sqlite:////tmp/amo_storage.db
; Redis Configurations
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_DB=0
; Service ID for data parcel
SERVICE_ID=00000001
[AuthConfig]
ISSUER=amo-storage
ALGORITHM=HS256
SECRET=your-sercret
[CephConfig]
HOST=127.0.0.1
PORT=7480
BUCKET_NAME=amo
[AmoBlockchainNodeConfig]
HOST=127.0.0.1
PORT=26657
To get more information about key.json
, read this section
NOTE: Before starting AMO-Storage API server, redis
server daemon must be running.
To run AMO-Storage API server, some dependencies must be installed. Install via below command.
$ pip3 install -r requirements.txt
And then run amo-stroage service via below command.
$ python3 main.py
* running on http://127.0.0.1:5000
Also you can run amo-storage service on specific host, port and config directory path via below command. Config directory is the directory where the config.ini
and key.json
files are located.
$ python3 main.py --host {host} --port {port} --config_dir {path to config directory}
The operations which need authorization process like upload
, download
and remove
should be requested with authorization headers as below.
Parameter | Type | Description |
---|---|---|
X-Auth-Token |
string |
Required. Received ACCESS_TOKEN |
X-Public-Key |
string |
Required. User's public key (hex encoded) |
X-Signature |
string |
Required. Signed ACCESS_TOKEN (hex encoded) |
To acquire ACCESS_TOKEN
, client should have to send POST
request to auth
API with below data in the request body.
{"user": user_identity, "operation": operation_description"}
The user_indentity
is the user account address in AMO ecosystem and operation_description
is composed of operation's name and parcel id or hash data like described below.
/* for `download`, `remove` operations */
{"name": operation_name, "id": parcel_id"}
/* for `upload` operation only */
{"name": operation_name, "hash":data_body_256_hash"}
The operation_name
can be "upload"
, "download"
, "remove"
and should be lowercase.
inspect
operation is not included because inspect
operation should be requested without auth
. For more detail, see the Operation Description.
When the ACCESS_TOKEN
is acquired, requesting client can construct request headers for an authorization. The request header must contains X-Auth-Token
, X-Public-Key
, X-Signature
values. X-Auth-Token
should be ACCESS_TOKEN
value and X-Public-Key
should be user's public key and it must be hex encoded format. Finally, X-Signature
should be signed ACCESS_TOKEN
which is signed with user's private key and it must be hex encoded format. Then, a client should send a request with those headers included.
When error is occurred, server will return the proper HTTP error code with response body : {"error":{ERROR_MESSAGE}}
.
- Each request body is
JSON-encoded
format. - Each API's request body parameter is well defined on AMO storage Documents.
POST /api/{api_version}/auth
Parameter | Type | Description |
---|---|---|
user |
string |
Required. user_identity |
operation |
JSON object |
Required. operation_description |
{
"token": ACCESS_TOKEN
}
Status Code | Error Message | Description |
---|---|---|
400 | Reason why request body is invalid | Invalid request body |
405 | None | Invalid request method |
Auth Required
POST /api/{api_version}/parcels
Parameter | Type | Description |
---|---|---|
owner |
string |
Required. user_identity |
metadata |
JSON object |
Required. metadata |
data |
string |
Required. hex_encoded_binary_sequence |
Metadata field is a schemeless JSON form, but the owner
field must be included.
{
"owner": user_identity, // Mandantory field
...
}
{
"id": data_parcel_id
}
Status Code | Error Message | Description |
---|---|---|
400 | Reason why request body is invalid | Invalid request body |
401 | One or more required fields do not exist in the header | |
401 | Invalid token | |
401 | Token does not exist | |
401 | Token does not have permission to perform the operation | |
401 | Verification failed | |
405 | None | Invalid request method |
409 | Parcel ID parcel_id already exists |
|
500 | Error occurred on saving ownership and metadata | Database error |
500 | Ceph error message |
Auth Required
GET /api/{api_version}/parcels/{parcel_id}
Parameter | Type | Description |
---|
{
"id": data_parcel_id,
"owner": user_indentity,
"data": hex_encoded_binary_sequence
}
Status Code | Error Message | Description |
---|---|---|
400 | Reason why request body is invalid | Invalid request body |
401 | One or more required fields do not exist in the header | |
401 | Invalid token | |
401 | Token does not exist | |
401 | Token is only available to perform operation_name |
|
401 | Verification failed | |
401 | No permission to download data parcel | |
405 | None | Invalid request method |
500 | Ceph error message |
GET /api/{api_version}/parcels/{parcel_id}?key=metadata
Parameter | Type | Description |
---|
{
"id": data_parcel_id,
"owner": user_indentity,
"metadata": metadata
}
Status Code | Error Message | Description |
---|---|---|
404 | None | parcel_id does not exist |
405 | None | Invalid request method |
Auth Required
DELETE /api/{api_version}/parcels/{parcel_id}
Parameter | Type | Description |
---|
{}
Status Code | Error Message | Description |
---|---|---|
401 | One or more required fields do not exist in the header | |
401 | Invalid token | |
401 | Token does not exist | |
401 | Token does not have permission to perform the operation | |
401 | Verification failed | |
405 | Not allowed to remove parcel | |
410 | Parcel does not exist | |
500 | Error occurred on deleting ownership and metadata | Database error |
500 | Ceph error message |