Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add multi-cloud user guide #101

Merged
merged 11 commits into from
Sep 17, 2020
Merged
5 changes: 2 additions & 3 deletions content/guides/developer-guides/file-share/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ disableToc: false
tags: ["developer guide", "file share"]
---

This tutorial guides users and new contributors to get familiar with OpenSDS by installing a simple local cluster and enabling NFS to use file share feature. You can also use the ansible script to install automatically. See details in OpenSDS Local Cluster Installation through ansible.
This tutorial guides users and new contributors to get familiar with SODA by installing a simple local cluster and enabling NFS to use file share feature. You can also use the ansible script to install automatically. See details in SODA Local Cluster Installation through ansible.

### Prerequisite
Make sure you have all the following:
Expand All @@ -18,8 +18,7 @@ Make sure you have all the following:
### Step by Step Installation
There are two projects provided to user, if user only wants file share feature, they need to have github.com/sodafoundation/api, github.com/sodafoundation/controller, github.com/sodafoundation/dock and github.com/sodafoundation/dashboard (if you need GUI).

#### OpenSDS (Hotpot) FIXME-SODA
Bootstrap
#### SODA Bootstrap
First you need to download bootstrap script and run it locally with root access.
```
curl -sSL https://raw.githubusercontent.com/opensds/opensds/master/install/devsds/bootstrap.sh | sudo bash
Expand Down
111 changes: 111 additions & 0 deletions content/guides/developer-guides/multi-cloud/_index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
---
title: Multi Cloud
description: "User guide for Multi Cloud service."
weight: 10
disableToc: false
tags: ["user guide", "multicloud"]
---


## Multi-cloud Local Cluster Installation and Testing
Here are the steps to install SODA Multi Cloud service in a containerized local environment .

### Pre-config (Ubuntu 16.04/Ubuntu 18.04)
The installation is tested on Ubuntu 16.04 and Ubuntu 18.04, please make sure to use the right one.
Root user is suggested for the installation.

* packages

Install following packages:
```bash
apt-get install -y libltdl7 libseccomp2 git curl wget make
```

* docker

Install docker:
```bash
wget https://download.docker.com/linux/ubuntu/dists/xenial/pool/stable/amd64/docker-ce_18.03.1~ce-0~ubuntu_amd64.deb
dpkg -i docker-ce_18.03.1~ce-0~ubuntu_amd64.deb
```
* golang

Check golang version information:
SODA multi-cloud needs go version 1.13.x

Example:
```bash
root@proxy:~# go version
go version go1.13.9 linux/amd64
```
You can install golang by executing commands below:
[Using 1.13.9 here for example]
```bash
wget https://storage.googleapis.com/golang/go1.13.9.linux-amd64.tar.gz
tar -C /usr/local -xzf go1.13.9.linux-amd64.tar.gz
echo 'export PATH=$PATH:/usr/local/go/bin' >> /etc/profile
echo 'export GOPATH=$HOME/gopath' >> /etc/profile
source /etc/profile
```
* docker-compose

Download and install the latest version of Docker Compose:
```bash
curl -L "https://github.com/docker/compose/releases/download/1.22.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose

# docker-compose --version
docker-compose version 1.22.0, build 1719ceb
```

### Install and start SODA Multi-cloud Server
Download the source code:
```bash
mkdir -p /root/gopath/src/github.com/sodafoundation/
git clone https://github.com/sodafoundation/multi-cloud.git
cd /root/gopath/src/github.com/sodafoundation/multi-cloud
```

Make and build the docker image
```bash
make docker
```

Execute `docker-compose up -d` to start multi-cloud local cluster, the successful result is supposed to be as follows:
```bash
# docker-compose up -d

Creating redis ... done
Creating multi-cloud_block_1_87f15fb3fcf4 ... done
Creating tidb ... done
Creating multi-cloud_zookeeper_1_77bbf2cb55a6 ... done
Creating multi-cloud_datastore_1_2341f44ebbba ... done
Creating multi-cloud_file_1_4885d7d3334d ... done
Creating multi-cloud_backend_1_5ab6fd6feed4 ... done
Creating multi-cloud_s3api_1_395c90ae45c3 ... done
Creating multi-cloud_api_1_437a019a9278 ... done
Creating multi-cloud_s3_1_5cf4e4514669 ... done
Creating multi-cloud_kafka_1_ff762483f0b6 ... done
Creating multi-cloud_dataflow_1_592ca1c6f730 ... done
Creating multi-cloud_datamover_1_390d343d0e7c ... done
```

To check the installation status run the command `docker ps`
```bash
# docker ps

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
bbe74a5359eb sodafoundation/multi-cloud-dataflow "/dataflow" 40 seconds ago Up 39 seconds multi-cloud_dataflow_1_8ce4de44c4a7
e18133da2d9f sodafoundation/multi-cloud-datamover "/datamover" 40 seconds ago Up 39 seconds multi-cloud_datamover_1_1ae5af80e61f
ec8052d3db15 wurstmeister/kafka:2.11-2.0.1 "start-kafka.sh" 41 seconds ago Up 39 seconds 0.0.0.0:9092->9092/tcp multi-cloud_kafka_1_74c5e392fd7d
5fb7560159b2 sodafoundation/multi-cloud-s3 "/initdb.sh" 41 seconds ago Up 40 seconds multi-cloud_s3_1_59cd2512722b
b28213c1ddec sodafoundation/multi-cloud-backend "/backend" 42 seconds ago Up 39 seconds multi-cloud_backend_1_a67a449c92a5
a47ec4ef770b sodafoundation/multi-cloud-file "/file" 42 seconds ago Up 40 seconds multi-cloud_file_1_cd84c1c9dad3
be2b170b2d68 sodafoundation/multi-cloud-api "/api" 42 seconds ago Up 41 seconds 0.0.0.0:8090->8090/tcp multi-cloud_s3api_1_4698613dfd2e
f0f2419b98db sodafoundation/multi-cloud-api "/api" 42 seconds ago Up 41 seconds 0.0.0.0:8089->8089/tcp multi-cloud_api_1_aacf7b9a5141
ad7a85f51343 wurstmeister/zookeeper "/bin/sh -c '/usr/sb…" 42 seconds ago Up 41 seconds 22/tcp, 2888/tcp, 3888/tcp, 0.0.0.0:2181->2181/tcp multi-cloud_zookeeper_1_68df196d327a
3539d1a91298 mongo "docker-entrypoint.s…" 42 seconds ago Up 41 seconds 0.0.0.0:27017->27017/tcp multi-cloud_datastore_1_a2f8fb0deb9d
0722d5174b8f pingcap/tidb:v2.1.16 "/tidb-server --stor…" 42 seconds ago Up 41 seconds 0.0.0.0:4000->4000/tcp, 0.0.0.0:10080->10080/tcp tidb
e03e03d88dcb sodafoundation/multi-cloud-block "/block" 42 seconds ago Up 40 seconds multi-cloud_block_1_7adafa5c5b8d
edc2036f5e1f redis "docker-entrypoint.s…" 42 seconds ago Up 41 seconds 0.0.0.0:6379->6379/tcp redis
```
Original file line number Diff line number Diff line change
Expand Up @@ -7,26 +7,26 @@ tags: ["developer guide", "object lifecycle"]
---
## Overview

This guide is intended for Object Lifecycle feature developers. It provides detailed steps to write new API using OpenSDS. To understand the feature and requirements in detail, please refer to the Object Lifecycle Management [user guide](/guides/user-guides/object-lifecycle-management/) and Object Lifecycle [design spec](https://github.com/opensds/design-specs/blob/master/specs/capri/object_lifecycle_management.md).
This guide is intended for Object Lifecycle feature developers. It provides detailed steps to write new API using SODA. To understand the feature and requirements in detail, please refer to the Object Lifecycle Management [user guide](/guides/user-guides/object-lifecycle-management/) and Object Lifecycle [design spec](https://github.com/opensds/design-specs/blob/master/specs/capri/object_lifecycle_management.md).
rajat-soda marked this conversation as resolved.
Show resolved Hide resolved


## APIs
APIs are a set of functions and procedures that allow for the creation of applications that access data and features of other applications, services or operating system. In OpenSDS lifecycle management feature we are using APIs to communicate with different storage backend.
APIs are a set of functions and procedures that allow for the creation of applications that access data and features of other applications, services or operating system. In SODA lifecycle management feature we are using APIs to communicate with different storage backend.


### Write new API
Before starting to write the APIs, it is required to have basic understanding of XML and JSON syntax.

#### There are three major steps involved:
a) All APIs go into the 's3' module of the OpenSDS multi-cloud package, i,e. opensds/multi-cloud/api/pkg/s3 directory
a) All APIs go into the 's3' module of the SODA multi-cloud package, i,e. opensds/multi-cloud/api/pkg/s3 directory
rajat-soda marked this conversation as resolved.
Show resolved Hide resolved
=======
## Write new API
Before starting to write the APIs, it is required to have basic understanding of XML and JSON syntax.

## How to write the APIs

### There are three major steps involved:
a) All APIs goes into the 's3' module of the OpenSDS multi-cloud package, i,e. opensds/multi-cloud/api/pkg/s3 directory
a) All APIs goes into the 's3' module of the SODA multi-cloud package, i,e. opensds/multi-cloud/api/pkg/s3 directory
rajat-soda marked this conversation as resolved.
Show resolved Hide resolved
b) The corresponding router for the API are defined same directory.
c) For every API there must be a function defined in proto file.
Example of adding API router (router.go):
Expand Down
26 changes: 13 additions & 13 deletions content/guides/user-guides/file-share/_index.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
---
title: File Share feature User Guide
title: File Share
description: "User guide for File Share feature."
weight: 10
disableToc: false
tags: ["user guide", "file share"]
---
## File Share Introduction
File share in opensds facilitates the user to create file share, set access permission for that file share and user can also create snapshot. Opensds provides both CLI and Dashboard to operate or play with file share feature.
Goals of file share in opensds are
Facilitating File Share Service by providing Standard API to manage multiple vendors, simplify File Share API definition
File share across the users based on access capability
File share facilitates with profiling
File share in SODA facilitates the user to create file share, set access permission for that file share and user can also create snapshot. SODA provides both CLI and Dashboard to operate or play with file share feature.
Goals of file share in SODA are
- Facilitating File Share Service by providing Standard API to manage multiple vendors, simplify File Share API definition
- File share across the users based on access capability
- File share facilitates with profiling
Scope for now is to support only NFS, SMB file share protocols

### Install file share feature with opensds
Follow the wiki link to install opensds https://github.com/sodafoundation/api/wiki/SODA-Projects-Cluster-Installation-through-Ansible
### Install file share feature with SODA
Follow the wiki link to install SODA https://github.com/sodafoundation/api/wiki/SODA-Projects-Cluster-Installation-through-Ansible

Make sure backend nfs is enable to use file share feature
###### NFS
Expand All @@ -25,8 +25,8 @@ enabled_backends: nfs

## Set the environment to start use of file share feature

### OpenSDS CLI
Firstly configure opensds CLI tool:
### SODA CLI
Firstly configure SODA CLI tool:
```bash
sudo cp /opt/opensds-hotpot-linux-amd64/bin/osdsctl /usr/local/bin/

Expand Down Expand Up @@ -75,15 +75,15 @@ Create snapshot of file share
osdsctl fileshare snapshot create <fileshare_id> -n <snapshot_name> -d "description of file share snapshot"
```

### OpenSDS UI
OpenSDS UI dashboard is available at `http://{your_host_ip}:8088`, please login the dashboard using the default admin credentials: `admin/opensds@123`. Create `tenant`, `user`, and `profiles` as admin.
### SODA UI
SODA UI dashboard is available at `http://{your_host_ip}:8088`, please login the dashboard using the default admin credentials: `admin/opensds@123`. Create `tenant`, `user`, and `profiles` as admin.

Logout of the dashboard as admin and login the dashboard again as a non-admin user to manage storage resource

#### File Share Services
* Create a file share

Post login into the OpenSDS UI, File share can be created from: Resource -> File share -> Create
Post login into the SODA UI, File share can be created from: Resource -> File share -> Create
![Fileshare create menu](fileshare_menu.PNG?raw=true)

* Create snapshot
Expand Down
26 changes: 13 additions & 13 deletions content/guides/user-guides/object-lifecycle-management/_index.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,21 @@
---
title: Object Lifecycle Management
title: Object Lifecycle
description: "User guide for Object Lifecycle Management."
weight: 40
disableToc: false
tags: ["user guide", "object lifecycle"]
---

## Introduction to Object lifecycle
OpenSDS is aimed at addressing the storage integration challenges of both the Cloud Native environment and traditional IT environment. OpenSDS multi-cloud allows user to manage distributed cloud environment on a single platform. Object lifecycle management feature gives freedom to user to manage buckets and objects in the cloud using OpenSDS interface.
SODA is aimed at addressing the storage integration challenges of both the Cloud Native environment and traditional IT environment. SODA multi-cloud allows user to manage distributed cloud environment on a single platform. Object lifecycle management feature gives freedom to user to manage buckets and objects in the cloud using SODA interface.

## Getting Started!
This is a guide that shows how to install, configure, and use Object lifecycle management feature in a simple OpenSDS setup.
By following this guide you will be able to install OpenSDS multi-cloud with creating backends and buckets, uploading object and setting up the lifecycle for those objects.
This is a guide that shows how to install, configure, and use Object lifecycle management feature in a simple SODA setup.
By following this guide you will be able to install SODA multi-cloud with creating backends and buckets, uploading object and setting up the lifecycle for those objects.

## Installing Opensds multi-cloud
## Installing SODA multi-cloud

### Download opensds-installer code
### Download SODA-installer code

```cassandraql
git clone https://github.com/opensds/opensds-installer.git
rajat-soda marked this conversation as resolved.
Show resolved Hide resolved
Expand All @@ -35,16 +35,16 @@ gelato_repo_branch: master
```
Detailed instruction is in this link

- [OpenSDS Local Cluster Installation through Ansible On Ubuntu](https://github.com/opensds/opensds/wiki/OpenSDS-Cluster-Installation-through-Ansible) (Recommended)
- [SODA Local Cluster Installation through Ansible On Ubuntu](https://github.com/opensds/opensds/wiki/OpenSDS-Cluster-Installation-through-Ansible) (Recommended)

#### Export required OpenSDS environment variables
#### Export required SODA environment variables
```
export HOST_IP={your_real_host_ip}
export OPENSDS_ENDPOINT=http://{{ apiserver_cluster_ip }}:50040
export OPENSDS_AUTH_STRATEGY=keystone
export OS_AUTH_URL=http://{{ authchecker_cluster_ip }}/identity
```
#### Run OpenSDS multi-cloud in docker container
#### Run SODA multi-cloud in docker container
```
docker-compose up -d
docker ps
Expand All @@ -62,9 +62,9 @@ e370acd6c922 mongo "docker-entrypoint.s…"
f139d2e383ab opensdsio/multi-cloud-s3 "/s3" 2 minutes ago Up 2 minutes multi-cloud_s3_1_3c7878089fa4
```

## Open OpenSDS Object lifecycle management UI
## Open SODA Object lifecycle management UI

OpenSDS Object lifecycle management feature specific UI can be accessed via
SODA Object lifecycle management feature specific UI can be accessed via

`http://{your_host_ip}:8088/#/home`

Expand All @@ -79,7 +79,7 @@ Click on (+) for registering a storage backend

### Create bucket and upload the object
After creating new backend follow the steps given below to create a bucket and upload an object in that bucket:
1. Launch Resource from OpenSDS home page
1. Launch Resource from SODA home page

2. Create a new bucket with appropriate backend

Expand All @@ -103,4 +103,4 @@ After configuring lifecycle in dashboard, a routine scheduler will run and it wi
1. How to check if the lifecycle rule is applied on bucket/object ?

Ans:
OpenSDS does not create the rule in the cloud backend (CSP). Once the bucket/object satisfies the rule, OpenSDS will automatically call cloud API and the action will be performed. If there are any errors or issues in rule, user will know at the time of lifecycle action.
SODA does not create the rule in the cloud backend (CSP). Once the bucket/object satisfies the rule, SODA will automatically call cloud API and the action will be performed. If there are any errors or issues in rule, user will know at the time of lifecycle action.
Loading