ocs-to-ocs api server design

This PR adds the design for API server that would help
Consumer clusters to communicate with OCS Provider cluster.

Signed-off-by: Santosh Pillai <sapillai@redhat.com>

docs/design/ocs-to-ocs-api-server.md

@@ -0,0 +1,162 @@
+# OCS to OCSs : API Server
+
+## Introduction
+This document defines the API server used for communication between OCS provider and OCS consumer clusters.
+
+
+## Tool
+Following tools can be used to create the API server:
+
+- REST (JSON over HTTP)
+- gRPC
+- OpenAPI3
+
+## Tool Comparison
+### gRPC vs REST
+
+- #### Performance:
+    - gRPC is more performant than REST due to smaller payloads (and other reasons).
+    - Current use case does not involve frequent requests between client and server. So performance aspect of the gRPC can be ignored.
+
+- #### Security:
+    - Both provide security with SSL/TLS support.
+    - gRPC endpoint can't be invoked via browser/postman.
+
+We will use gRPC as we are on k8s and gRPC has more advantages compared to REST
+
+
+## RPC Interface
+
+#### OCSProvider service
+
+``` protobuf
+syntax = "proto3";
+package provider;
+
+import "google/protobuf/descriptor.proto";
+import "google/protobuf/struct.proto";
+
+option go_package = "./;providerpb";
+
+
+// OCSProvider holds the RPC methods that the OCS consumer can use to communicate with remote OCS provider cluster
+service OCSProvider {
+  // OnBoardConsumer RPC call to onboard a new OCS consumer cluster.
+  rpc OnBoardConsumer (OnBoardConsumerRequest)
+  returns (OnBoardConsumerResponse) {}
+  // OffboardConsumer RPC call to delete the StorageConsumer CR.
+  rpc OffboardConsumer (OffboardConsumerRequest)
+  returns (OffboardConsumerResponse) {}
+  // UpdateCapacity PRC call to increase or decrease the storage pool size
+  rpc UpdateCapacity(UpdateCapacityRequest)
+  returns (UpdateCapacityResponse)
+}
+```
+
+#### OnboardConsumer
+- Read the `status.state` of the StorageConsumer CR:
+  - if `status` is `onboarding`, then update the `spec.Capacity` in the CR. Return `UNAVAILABLE`.
+  - if `status` is `Provisioning`, return `UNAVAILABLE`.
+  - if `status` is `Ready`, fetch the `rbdPoolName` and the`cephUser` details from the CR status. Generate and return json connection details.
+
+**Note**: When `UNAVAILABLE` status is returned, the consumer controller should requeue (with interval) the reconciler to call the onboarding API again to check the StorageConsumer CR status.
+
+
+```protobuf
+message OnBoardConsumerRequest{
+    // K8s UID (UUID) of the consumer cluster
+    string storageConsumerUUID =1;
+    // capacity is a valid k8s resource quantity
+    string capacity = 2;
+}
+
+message OnBoardConsumerResponse{
+    // data contains the json blob to be used by the consumer cluster to connect with the provider cluster
+   google.protobuf.Struct data = 1;
+}
+
+```
+
+##### Response Scheme
+
+
+| Condition | gRPC Code | Description | Recovery |
+| -------- | -------- | -------- | -------- |
+| Success    | 0  OK     | Request Successful. StorageConsumer CR is updated, rook resources are created and CR status is updated successfully.    | N/A |
+| Awaiting Ceph resources    | 14 UNAVAILABLE     | Waiting for the ceph resourcs to be created and updated in the StoragConsumer CR status    | N/A |
+| Invalid ID | 3 INVALID_ARGUMENT | Request failed to invalid Consumer ID | Contact the Provider Cluster Admin to verify the StorageConsumer ID
+| Error | 13 INTERNAL | Request failed due to some error in provider cluster | Contact Provider Cluster Admin
+
+
+#### OffboardConsumer
+- Make a delete request on the StorageConsumer CR and return OK.
+
+```protobuf
+message OffBoardConsumerRequest{
+    // K8s UID (UUID) of the consumer cluster
+    string storageConsumerUUID =1;
+}
+
+message OffBoardConsumerResponse{
+
+}
+
+```
+
+##### Response Scheme
+| Condition | gRPC Code | Description | Recovery |
+| -------- | -------- | -------- | -------- |
+| Success    | 0  OK     | Request Successful     | N/A |
+| Invalid ID | 3 INVALID_ARGUMENT | Request failed due to invalid Consumer ID | Contact the Provider Cluster Admin to verify the StorageConsumer ID
+| Error | 13 INTERNAL | Request failed due to some error in provider cluster | Contact Provider Cluster Admin
+
+
+
+#### UpdateCapacity
+- Read the `status.state` of the StorageConsumer CR:
+  - if Status is `Ready`, then update the `spec.Capacity` and return OK
+
+```proto=
+
+message UpdateCapacityRequest{
+    // K8s UID (UUID) of the consumer cluster
+    string storageConsumerUUID =1;
+    // capacity is a valid k8s resource quantity
+    string capacity = 2;
+}
+
+message UpdateCapacityResponse{
+
+}
+```
+
+##### Response Scheme
+| Condition | gRPC Code | Description | Recovery |
+| -------- | -------- | -------- | -------- |
+| Success    | 0  OK     | Request Successful     | N/A |
+| Invalid ID | 3 INVALID_ARGUMENT | Request failed to invalid Consumer ID | Contact the Provider Cluster Admin to verify the StorageConsumer ID
+| Invalid Capacity | 3 INVALID_ARGUMENT | Request failed due to invalid capacity | Contact the Provider Cluster Admin to get the valid capacity value
+| Error | 13 INTERNAL | Request failed due to some error in the Provider Cluster | Contact Provider Cluster Admin
+
+
+
+## PROJECT STRUCTURE
+- The API server would be part of the OCS Operator.
+- It would be deployed as separate deployment on the OCS cluster only after user has enabled it in the StorageCluster CR via `spec.allowRemoteStorageConsumers`
+
+```
+.
+├── services
+    ├── provider
+        ├── pb
+            ├── provider.proto
+            ├── provider.pb.go
+            ├── README.md
+        ├── client
+            ├── client.go
+        ├── server
+            ├── server.go
+
+```
+
+## API SECURITY