Merge pull request #1617 from cosmos/adrian/lcd_spec_final

LCD and API Specification / Documentation
cosmos · Jul 19, 2018 · 27840ab · 27840ab
2 parents d2f70ec + 42c2712
commit 27840ab
Show file tree

Hide file tree

Showing 29 changed files with 1,121 additions and 0 deletions.
diff --git a/docs/light/api.md b/docs/light/api.md
diff --git a/docs/light/getting_started.md b/docs/light/getting_started.md
@@ -0,0 +1,40 @@
+# Getting Started
+
+To start a rest server, we need to specify the following parameters:
+| Parameter   | Type      | Default                 | Required | Description                                          |
+| ----------- | --------- | ----------------------- | -------- | ---------------------------------------------------- |
+| chain-id    | string    | null                    | true     | chain id of the full node to connect                 |
+| node        | URL       | "tcp://localhost:46657" | true     | address of the full node to connect                  |
+| laddr       | URL       | "tcp://localhost:1317"  | true     | address to run the rest server on                    |
+| trust-node  | bool      | "false"                 | true     | Whether this LCD is connected to a trusted full node |
+| trust-store | DIRECTORY | "$HOME/.lcd"            | false    | directory for save checkpoints and validator sets    |
+
+Sample command:
+
+```bash
+gaiacli light-client --chain-id=test --laddr=tcp://localhost:1317  --node tcp://localhost:46657 --trust-node=false
+```
+
+## Gaia Light Use Cases
+
+LCD could be very helpful for related service providers. For a wallet service provider, LCD could
+make transaction faster and more reliable in the following cases.
+
+### Create an account
+
+![deposit](pics/create-account.png)
+
+First you need to get a new seed phrase :[get-seed](api.md#keysseed---get)
+
+After having new seed, you could generate a new account with it : [keys](api.md#keys---post)
+
+### Transfer a token
+
+![transfer](pics/transfer-tokens.png)
+
+The first step is to build an asset transfer transaction. Here we can post all necessary parameters
+to /create_transfer to get the unsigned transaction byte array. Refer to this link for detailed
+operation: [build transaction](api.md#create_transfer---post)
+
+Then sign the returned transaction byte array with users' private key. Finally broadcast the signed
+transaction. Refer to this link for how to broadcast the signed transaction: [broadcast transaction](api.md#create_transfer---post)
diff --git a/docs/light/load_balancer.md b/docs/light/load_balancer.md
@@ -0,0 +1,203 @@
+# Load Balancing Module - WIP
+
+The LCD will be an important bridge between service providers and cosmos blockchain network. Suppose
+a service provider wants to monitor token information for millions of accounts. Then it has to keep
+sending a large mount of requests to LCD to query token information. As a result, LCD will send huge
+requests to full node to get token information and necessary proof which will cost full node much
+computing and bandwidth resource. Too many requests to a single full node may result in some bad
+situations:
+
+```text
+1. The full node crash possibility increases.
+2. The reply delay increases.
+3. The system reliability will decrease.
+4. As the full node may belong to other people or associates, they may deny too frequent access from a single client.
+```
+
+It is very urgent to solve this problems. Here we consider to import load balancing into LCD. By the
+help of load balancing, LCD can distribute millions of requests to a set of full nodes. Thus the
+load of each full node won't be too heavy and the unavailable full nodes will be wiped out of query
+list. In addition, the system reliability will increase.
+
+## Design
+
+This module need combine with client to realize the real load balancing. It can embed the
+[HTTP Client](https://github.com/tendermint/tendermint/rpc/lib/client/httpclient.go). In other 
+words，we realise the new httpclient based on `HTTP`.
+
+```go
+type HTTPLoadBalancer struct {
+    rpcs   map[string]*rpcclient.JSONRPCClient
+    *WSEvents
+}
+```
+
+## The Diagram of LCD RPC WorkFlow with LoadBalance
+
+![The Diagram of LCD RPC WorkFlow](pics/loadbalanceDiagram.png)
+
+In the above sequence diagram, application calls the `Request()`, and LCD finally call the
+`HTTP.Request()` through the SecureClient `Wrapper`. In every `HTTP.Request()`, `Getclient()`
+selects the current working rpcclient by the load balancing algorithm,then run the
+`JSONRPCClient.Call()` to request from the Full Node, finally `UpdateClient()` updates the weight of
+ the current rpcclient according to the status that is returned by the full node. The `GetAddr()`
+ and `UpdateAddrWeight()` are realized  in the load balancing module.
+
+There are some abilities to do:
+
+* Add the Remote Address
+* Delete the Remote Address
+* Update the weights of the addresses
+
+## Load balancing Strategies
+
+We can design some strategies like nginx to combine the different load balancing algorithms to get
+the final remote. We can also get the status of the remote server to add or delete the addresses and
+ update weights of the addresses.
+
+In a word，it can make the entire LCD work more effective in actual conditions.
+We are working this module independently in this [Github Repository](https://github.com/MrXJC/GoLoadBalance).
+
+## Interface And Type
+
+### Balancer
+
+This interface `Balancer`is the core of the package. Every load balancing algorithm should realize
+it,and it defined two interfaces.
+
+* `init` initialize the balancer, assigns the variables which `DoBalance` needs.
+* `DoBalance` load balance the full node addresses according to the current situation.
+
+```go
+package balance
+
+type  Balancer interface {
+    init(NodeAddrs)
+    DoBalance(NodeAddrs) (*NodeAddr,int,error)
+}
+```
+
+### NodeAddr
+
+* host: ip address
+* port: the number of port
+* weight: the weight of this full node address,default:1
+
+This NodeAddr is the base struct of the address.
+
+```go
+type NodeAddr struct{
+    host string
+    port int
+    weight int
+}
+
+func (p *NodeAddr) GetHost() string
+
+func (p *NodeAddr) GetPort() int
+
+func (p *NodeAddr) GetWeight() int
+
+func (p *NodeAddr) updateWeight(weight int)
+```
+
+The `weight` is the important factor that schedules which full node the LCD calls. The weight can be
+changed by the information from the full node. So we have the function `updateWegiht`.
+
+### NodeAddrs
+
+>in `balance/types.go`
+
+`NodeAddrs` is the list of the full node address. This is the member variable in the 
+BalanceManager(`BalancerMgr`).
+
+```go
+type NodeAddrs []*NodeAddr
+```
+
+## Load Balancing Algorithm
+
+### Random
+
+>in `balance/random.go`
+
+Random algorithm selects a remote address randomly to process the request. The probability of them
+being selected is the same.
+
+### RandomWeight
+
+>in `balance/random.go`
+
+RandomWeight Algorithm also selects a remote address randomly to process the request. But the higher
+the weight, the greater the probability.
+
+### RoundRobin
+
+>in `balance/roundrobin.go`
+
+RoundRobin Algorithm selects a remote address orderly. Every remote address have the same 
+probability to be selected.
+
+### RoundRobinWeight
+
+>in `balance/roundrobin.go`
+
+RoundRobinWeight Algorthm selects a remote address orderly. But every remote address have different 
+probability to be selected which are determined by their weight.
+
+### Hash
+
+//TODO
+
+## Load Balancing Manager
+
+### BalanceMgr
+
+>in `balance/manager.go`
+
+* addrs: the set of the remote full node addresses
+* balancers: map the string of balancer name to the specific balancer
+* change: record whether the machine reinitialize after the `addrs` changes
+
+`BalanceMgr` is the manager of many balancer. It is the access of load balancing. Its main function 
+is to maintain the `NodeAddrs` and to call the specific load balancing algorithm above.
+
+```go
+type BalanceMgr struct{
+    addrs NodeAddrs
+    balancers map[string]Balancer
+    change map[string]bool
+}
+
+func (p *BalanceMgr) RegisterBalancer(name string,balancer Balancer)
+
+func (p *BalanceMgr) updateBalancer(name string)
+
+func (p *BalanceMgr) AddNodeAddr(addr *NodeAddr)
+
+func (p *BalanceMgr) DeleteNodeAddr(i int)
+
+func (p *BalanceMgr) UpdateWeightNodeAddr(i int,weight int)
+
+func (p *BalanceMgr) GetAddr(name string)(*NodeAddr,int,error) {
+    // if addrs change,update the balancer which we use.
+    if p.change[name]{
+        p.updateBalancer(name)
+    }
+
+    // get the balancer by name
+    balancer := p.balancers[name]
+
+    // use the load balancing algorithm
+    addr,index,err := balancer.DoBalance(p.addrs)
+
+    return addr,index,err
+}
+```
+
+* `RegisterBalancer`: register the basic balancer implementing the `Balancer` interface and initialize them.
+* `updateBalancer`: update the specific balancer after the `addrs` change.
+* `AddNodeAddr`: add the remote address and set all the values of the `change` to true.
+* `DeleteNodeAddr`: delete the remote address and set all the values of the `change` to true.
+* `UpdateWeightNodeAddr`: update the weight of the remote address and set all the values of the `change` to true.
+* `GetAddr`:select the address by the balancer the `name` decides.  
diff --git a/docs/light/pics/C2H.png b/docs/light/pics/C2H.png
diff --git a/docs/light/pics/H2C.png b/docs/light/pics/H2C.png
diff --git a/docs/light/pics/MA.png b/docs/light/pics/MA.png
diff --git a/docs/light/pics/absence1.png b/docs/light/pics/absence1.png
diff --git a/docs/light/pics/absence2.png b/docs/light/pics/absence2.png
diff --git a/docs/light/pics/absence3.png b/docs/light/pics/absence3.png
diff --git a/docs/light/pics/architecture.png b/docs/light/pics/architecture.png
diff --git a/docs/light/pics/changeProcess.png b/docs/light/pics/changeProcess.png
diff --git a/docs/light/pics/commitValidation.png b/docs/light/pics/commitValidation.png
diff --git a/docs/light/pics/create-account.png b/docs/light/pics/create-account.png
diff --git a/docs/light/pics/deposit.png b/docs/light/pics/deposit.png
diff --git a/docs/light/pics/existProof.png b/docs/light/pics/existProof.png
diff --git a/docs/light/pics/high-level.png b/docs/light/pics/high-level.png
diff --git a/docs/light/pics/light-client-architecture.png b/docs/light/pics/light-client-architecture.png
diff --git a/docs/light/pics/loadbalanceDiagram.png b/docs/light/pics/loadbalanceDiagram.png
diff --git a/docs/light/pics/simpleMerkleTree.png b/docs/light/pics/simpleMerkleTree.png
diff --git a/docs/light/pics/substoreProof.png b/docs/light/pics/substoreProof.png
diff --git a/docs/light/pics/transfer-tokens.png b/docs/light/pics/transfer-tokens.png
diff --git a/docs/light/pics/transfer.png b/docs/light/pics/transfer.png
diff --git a/docs/light/pics/trustPropagate.png b/docs/light/pics/trustPropagate.png
diff --git a/docs/light/pics/updateValidatorToHeight.png b/docs/light/pics/updateValidatorToHeight.png
diff --git a/docs/light/pics/validatorSetChange.png b/docs/light/pics/validatorSetChange.png
diff --git a/docs/light/pics/withdraw.png b/docs/light/pics/withdraw.png
diff --git a/docs/light/readme.md b/docs/light/readme.md
@@ -0,0 +1,100 @@
+# Cosmos-Sdk Light Client
+
+## Introduction
+
+A light client allows clients, such as mobile phones, to receive proofs of the state of the
+blockchain from any full node. Light clients do not have to trust any full node, since they are able
+to verify any proof they receive and hence full nodes cannot lie about the state of the network.
+
+A light client can provide the same security as a full node with the minimal requirements on
+bandwidth, computing and storage resource. Besides, it can also provide modular functionality
+according to users' configuration. These fantastic features allow developers to build fully secure,
+efficient and usable mobile apps, websites or any other applications without deploying or
+maintaining any full blockchain nodes.
+
+LCD will be used in the Cosmos Hub, the first Hub in the Cosmos network.
+
+## Contents
+
+1. [**Overview**](##Overview)
+2. [**Get Started**](getting_started.md)
+3. [**API**](api.md)
+4. [**Specifications**](hspecification.md)
+
+## Overview
+
+### What is a Light Client
+
+The LCD is split into two separate components. The first component is generic for any Tendermint
+based application. It handles the security and connectivity aspects of following the header chain
+and verify proofs from full nodes against locally trusted validator set. Furthermore it exposes
+exactly the same API as any Tendermint Core node. The second component is specific for the Cosmos
+Hub (Gaiad). It works as a query endpoint and exposes the application specific functionality, which
+can be arbitrary. All queries against the application state have to go through the query endpoint.
+The advantage of the query endpoint is that it can verify the proofs that the application returns.
+
+### High-Level Architecture
+
+An application developer that would like to build a third party integration can ship his application
+with the LCD for the Cosmos Hub (or any other zone) and only needs to initialise it. Afterwards his
+application can interact with the zone as if it was running against a full node.
+
+![high-level](pics/high-level.png)
+
+An application developer that wants to build an third party application for the Cosmos Hub (or any
+other zone) should build it against it's canonical API. That API is a combination of multiple parts.
+All zones have to expose ICS0 (TendermintAPI). Beyond that any zone is free to choose any
+combination of module APIs, depending on which modules the state machine uses. The Cosmos Hub will
+initially support ICS0 (TendermintAPI), ICS1 (KeyAPI), ICS20 (TokenAPI), ICS21 (StakingAPI) and
+ICS22 (GovernanceAPI).
+
+All applications are expected to only run against the LCD. The LCD is the only piece of software
+that offers stability guarantees around the zone API.
+
+### Comparision
+
+A full node of ABCI is different from its light client in the following ways:
+
+|| Full Node | LCD | Description|
+|-| ------------- | ----- | -------------- |
+| Execute and verify transactions|Yes|No|Full node will execute and verify all transactions while LCD won't|
+| Verify and save blocks|Yes|No|Full node will verify and save all blocks while LCD won't|
+| Participate consensus| Yes|No|Only when the full node is a validtor, it will participate consensus. LCD nodes never participate consensus|
+| Bandwidth cost|Huge|Little|Full node will receive all blocks. if the bandwidth is limited, it will fall behind the main network. What's more, if it happens to be a validator,it will slow down the consensus process. LCD requires little bandwidth. Only when serving local request, it will cost bandwidth|
+| Computing resource|Huge|Little|Full node will execute all transactions and verify all blocks which require much computing resource|
+| Storage resource|Huge|Little|Full node will save all blocks and ABCI states. LCD just saves validator sets and some checkpoints|
+| Power consume|Huge|Little|Full nodes have to be deployed on machines which have high performance and will be running all the time. So power consume will be huge. LCD can be deployed on the same machines as users' applications, or on independent machines but with poor performance. Besides, LCD can be shutdown anytime when necessary. So LCD only consume very little power, even mobile devices can meet the power requirement|
+| Provide APIs|All cosmos APIs|Modular APIs|Full node supports all cosmos APIs. LCD provides modular APIs according to users' configuration|
+| Secuity level| High|High|Full node will verify all transactions and blocks by itself. LCD can't do this, but it can query any data from other full nodes and verify the data independently. So both full node and LCD don't need to trust any third nodes, they all can achieve high security|
+
+According to the above table, LCD can meet all users' functionality and security requirements, but 
+only requires little resource on bandwidth, computing, storage and power.
+
+## How does LCD achieve high security?
+
+### Trusted validator set
+
+The base design philosophy of lcd follows the two rules:
+
+1. **Doesn't trust any blockchain nodes, including validator nodes and other full nodes**
+2. **Only trusts the whole validator set**
+
+The original trusted validator set should be prepositioned into its trust store, usually this
+validator set comes from genesis file. During running time, if LCD detects different validator set,
+it will verify it and save new validated validator set to trust store.
+
+![validator-set-change](pics/validatorSetChange.png)
+
+### Trust propagation
+
+From the above section, we come to know how to get trusted validator set and how lcd keeps track of
+validator set evolution. Validator set is the foundation of trust, and the trust can propagate to
+other blockchain data, such as block and transaction. The propagate architecture is shown as
+follows:
+
+![change-process](pics/trustPropagate.png)
+
+In general, by trusted validator set, LCD can verify each block commit which contains all pre-commit
+data and block header data. Then the block hash, data hash and appHash are trusted. Based on this
+and merkle proof, all transactions data and ABCI states can be verified too. Detailed implementation
+will be posted on technical specification.