diff --git a/docs/CONTRIBUTING.md b/docs/CONTRIBUTING.md index 0fe1e29e31b..b536f63bfb6 100644 --- a/docs/CONTRIBUTING.md +++ b/docs/CONTRIBUTING.md @@ -78,6 +78,14 @@ the `lf-sandbox` you should be ready to set up your local development [environment](dev-setup/devenv.md). +Next, try [building the project](dev-setup/build.md) in your local development +environment to ensure that everything is set up correctly. + +[Logging control](Setup/logging-control.md) describes how to tweak the logging +levels of various components within the Fabric. Finally, every source file +needs to include a [license header](dev-setup/headers.txt): modified to include +a copyright statement for the principle author(s). + ## What makes a good change request? * One change at a time. Not five, not three, not ten. One and only one. Why? @@ -138,8 +146,24 @@ getting it merged and adds more work for you - to remediate the merge conflicts. ## Coding guidelines -Be sure to check out the language-specific [style guides](Style-guides/go-style.md) -before making any changes. This will ensure a smoother review. +Be sure to check out the language-specific +[style guides](Style-guides/go-style.md) before making any changes. This will +ensure a smoother review. + +## Communication + +We use [RocketChat](https://chat.hyperledger.org/) for communication and +Google Hangouts™ for screen sharing between developers. Our development +planning and prioritization is done in [JIRA](https://jira.hyperledger.org), +and we take longer running discussions/decisions to the +[mailing list](http://lists.hyperledger.org/mailman/listinfo/hyperledger-fabric). + +## Maintainers + +The project's [maintainers](MAINTAINERS.md) are responsible for reviewing and +merging all patches submitted for review and they guide the over-all technical +direction of the project within the guidelines established by the Hyperledger +Project's Technical Steering Committee (TSC). ### Becoming a maintainer diff --git a/docs/arch-deep-dive.md b/docs/arch-deep-dive.md new file mode 100644 index 00000000000..887dada7abc --- /dev/null +++ b/docs/arch-deep-dive.md @@ -0,0 +1,369 @@ +This page documents the architecture of a blockchain infrastructure with the roles of a blockchain node separated into roles of *peers* (who maintain state/ledger) and *orderers* (who consent on the order of transactions included in the ledger). In common blockchain architectures (including Hyperledger Fabric v0.6 and earlier) these roles are unified (cf. *validating peer* in Hyperledger Fabric v0.6). The architecture also introduces *endorsing peers* (endorsers), as special type of peers responsible for simulating execution and *endorsing* transactions (roughly corresponding to executing transactions in HL Fabric 0.6). + +The architecture has the following advantages compared to the design in which peers/orderers/endorsers are unified (e.g., HL Fabric v0.6). + +* **Chaincode trust flexibility.** The architecture separates *trust assumptions* for chaincodes (blockchain applications) from trust assumptions for ordering. In other words, the ordering service may be provided by one set of nodes (orderers) and tolerate some of them to fail or misbehave, and the endorsers may be different for each chaincode. + +* **Scalability.** As the endorser nodes responsible for particular chaincode are orthogonal to the orderers, the system may *scale* better than if these functions were done by the same nodes. In particular, this results when different chaincodes specify disjoint endorsers, which introduces a partitioning of chaincodes between endorsers and allows parallel chaincode execution (endorsement). Besides, chaincode execution, which can potentially be costly, is removed from the critical path of the ordering service. + +* **Confidentiality.** The architecture facilitates deployment of chaincodes that have *confidentiality* requirements with respect to the content and state updates of its transactions. + +* **Consensus modularity.** The architecture is *modular* and allows pluggable consensus (i.e., ordering service) implementations. + +This architecture drives the development of Hyperledger Fabric post-v0.6. As detailed below, some of its aspects are to be included in Hyperledger Fabric v1, whereas others are postponed to post-v1 versions of Hyperledger Fabric. + +## Table of contents + +**Part I: Elements of the architecture relevant to Hyperledger Fabric v1** + +1. System architecture +1. Basic workflow of transaction endorsement +1. Endorsement policies + + **Part II: Post-v1 elements of the architecture** + +1. Ledger checkpointing (pruning) + + +--- + +## 1. System architecture + +The blockchain is a distributed system consisting of many nodes that communicate with each other. The blockchain runs programs called chaincode, holds state and ledger data, and executes transactions. The chaincode is the central element as transactions are operations invoked on the chaincode. Transactions have to be "endorsed" and only endorsed transactions may be committed and have an effect on the state. There may exist one or more special chaincodes for management functions and parameters, collectively called *system chaincodes*. + + +### 1.1. Transactions + +Transactions may be of two types: + +* *Deploy transactions* create new chaincode and take a program as parameter. When a deploy transaction executes successfully, the chaincode has been installed "on" the blockchain. + +* *Invoke transactions* perform an operation in the context of previously deployed chaincode. An invoke transaction refers to a chaincode and to one of its provided functions. When successful, the chaincode executes the specified function - which may involve modifying the corresponding state, and returning an output. + +As described later, deploy transactions are special cases of invoke transactions, where a deploy transaction that creates new chaincode, corresponds to an invoke transaction on a system chaincode. + +**Remark:** *This document currently assumes that a transaction either creates new chaincode or invokes an operation provided by _one_ already deployed chaincode. This document does not yet describe: a) optimizations for query (read-only) transactions (included in v1), b) support for cross-chaincode transactions (post-v1 feature).* + +### 1.2. Blockchain datastructures + +#### 1.2.1. State + +The latest state of the blockchain (or, simply, *state*) is modeled as a versioned key/value store (KVS), where keys are names and values are arbitrary blobs. These entries are manipulated by the chaincodes (applications) running on the blockchain through `put` and `get` KVS-operations. The state is stored persistently and updates to the state are logged. Notice that versioned KVS is adopted as state model, an implementation may use actual KVSs, but also RDBMSs or any other solution. + +More formally, state `s` is modeled as an element of a mapping `K -> (V X N)`, where: + +* `K` is a set of keys +* `V` is a set of values +* `N` is an infinite ordered set of version numbers. Injective function `next: N -> N` takes an element of `N` and returns the next version number. + +Both `V` and `N` contain a special element `\bot`, which is in case of `N` the lowest element. Initially all keys are mapped to `(\bot,\bot)`. For `s(k)=(v,ver)` we denote `v` by `s(k).value`, and `ver` by `s(k).version`. + +KVS operations are modeled as follows: + +* `put(k,v)`, for `k\in K` and `v\in V`, takes the blockchain state `s` and changes it to `s'` such that `s'(k)=(v,next(s(k).version))` with `s'(k')=s(k')` for all `k'!=k`. +* `get(k)` returns `s(k)`. + +State is maintained by peers, but not by orderers and clients. + +**State partitioning.** Keys in the KVS can be recognized from their name to belong to a particular chaincode, in the sense that only transaction of a certain chaincode may modify the keys belonging to this chaincode. In principle, any chaincode can read the keys belonging to other chaincodes. *Support for cross-chaincode transactions, that modify the state belonging to two or more chaincodes is a post-v1 feature.* + + +#### 1.2.2 Ledger + +Ledger provides a verifiable history of all successful state changes (we talk about *valid* transactions) and unsuccessful attempts to change state (we talk about *invalid* transactions), occurring during the operation of the system. + +Ledger is constructed by the ordering service (see Sec 1.3.3) as a totally ordered hashchain of *blocks* of (valid or invalid) transactions. The hashchain imposes the total order of blocks in a ledger and each block contains an array of totally ordered transactions. This imposes total order across all transactions. + +Ledger is kept at all peers and, optionally, at a subset of orderers. In the context of an orderer we refer to the Ledger as to `OrdererLedger`, whereas in the context of a peer we refer to the ledger as to `PeerLedger`. `PeerLedger` differs from the `OrdererLedger` in that peers locally maintain a bitmask that tells apart valid transactions from invalid ones (see Section XX for more details). + +Peers may prune `PeerLedger` as described in Section XX (post-v1 feature). Orderers maintain `OrdererLedger` for fault-tolerance and availability (of the `PeerLedger`) and may decide to prune it at anytime, provided that properties of the ordering service (see Sec. 1.3.3) are maintained. + +The ledger allows peers to replay the history of all transactions and to reconstruct the state. Therefore, state as described in Sec 1.2.1 is an optional datastructure. + +### 1.3. Nodes + +Nodes are the communication entities of the blockchain. A "node" is only a logical function in the sense that multiple nodes of different types can run on the same physical server. What counts is how nodes are grouped in "trust domains" and associated to logical entities that control them. + +There are three types of nodes: + +1. **Client** or **submitting-client**: a client that submits an actual transaction-invocation to the endorsers, and broadcasts transaction-proposals to the ordering service. + +1. **Peer**: a node that commits transactions and maintains the state and a copy of the ledger (see Sec, 1.2). Besides, peers can have a special **endorser** role. + +1. **Ordering-service-node** or **orderer**: a node running the communication service that implements a delivery guarantee, such as atomic or total order broadcast. + +The types of nodes are explained next in more detail. + +#### 1.3.1. Client + +The client represents the entity that acts on behalf of an end-user. It must connect to a peer for communicating with the blockchain. The client may connect to any peer of its choice. Clients create and thereby invoke transactions. + +As detailed in Section 2, clients communicate with both peers and the ordering service. + +#### 1.3.2. Peer + +A peer receives ordered state updates in the form of *blocks* from the ordering service and maintain the state and the ledger. + +Peers can additionally take up a special role of an **endorsing peer**, or an **endorser**. The special function of an *endorsing peer* occurs with respect to a particular chaincode and consists in *endorsing* a transaction before it is committed. Every chaincode may specify an *endorsement policy* that may refer to a set of endorsing peers. The policy defines the necessary and sufficient conditions for a valid transaction endorsement (typically a set of endorsers' signatures), as described later in Sections 2 and 3. In the special case of deploy transactions that install new chaincode the (deployment) endorsement policy is specified as an endorsement policy of the system chaincode. + + +#### 1.3.3. Ordering service nodes (Orderers) + +The *orderers* form the *ordering service*, i.e., a communication fabric that provides delivery guarantees. The ordering service can be implemented in different ways: ranging from a centralized service (used e.g., in development and testing) to distributed protocols that target different network and node fault models. + +Ordering service provides a shared *communication channel* to clients and peers, offering a broadcast service for messages containing transactions. Clients connect to the channel and may broadcast messages on the channel which are then delivered to all peers. The channel supports *atomic* delivery of all messages, that is, message communication with total-order delivery and (implementation specific) reliability. In other words, the channel outputs the same messages to all connected peers and outputs them to all peers in the same logical order. This atomic communication guarantee is also called *total-order broadcast*, *atomic broadcast*, or *consensus* in the context of distributed systems. The communicated messages are the candidate transactions for inclusion in the blockchain state. + +**Partitioning (ordering service channels).** Ordering service may support multiple *channels* similar to the *topics* of a publish/subscribe (pub/sub) messaging system. Clients can connects to a given channel and can then send messages and obtain the messages that arrive. Channels can be thought of as partitions - clients connecting to one channel are unaware of the existence of other channels, but clients may connect to multiple channels. Even though some ordering service implementations included with Hyperledger Fabric v1 will support multiple channels, for simplicity of presentation, in the rest of this document, we assume ordering service consists of a single channel/topic. + +**Ordering service API.** Peers connect to the channel provided by the ordering service, via the interface provided by the ordering service. The ordering service API consists of two basic operations (more generally *asynchronous events*): + +**TODO** add the part of the API for fetching particular blocks under client/peer specified sequence numbers. + +* `broadcast(blob)`: a client calls this to broadcast an arbitrary message `blob` for dissemination over the channel. This is also called `request(blob)` in the BFT context, when sending a request to a service. + +* `deliver(seqno, prevhash, blob)`: the ordering service calls this on the peer to deliver the message `blob` with the specified non-negative integer sequence number (`seqno`) and hash of the most recently delivered blob (`prevhash`). In other words, it is an output event from the ordering service. `deliver()` is also sometimes called `notify()` in pub-sub systems or `commit()` in BFT systems. + +**Ledger and block formation.** The ledger (see also Sec. 1.2.2) contains all data output by the ordering service. In a nutshell, it is a sequence of `deliver(seqno, prevhash, blob)` events, which form a hash chain according to the computation of `prevhash` described before. + +Most of the time, for efficiency reasons, instead of outputting individual transactions (blobs), the ordering service will group (batch) the blobs and output *blocks* within a single `deliver` event. In this case, the ordering service must impose and convey a deterministic ordering of the blobs within each block. The number of blobs in a block may be chosen dynamically by an ordering service implementation. + +In the following, for ease of presentation, we define ordering service properties (rest of this subsection) and explain the workflow of transaction endorsement (Section 2) assuming one blob per `deliver` event. These are easily extended to blocks, assuming that a `deliver` event for a block corresponds to a sequence of individual `deliver` events for each blob within a block, according to the above mentioned deterministic ordering of blobs within a blocs. + +**Ordering service properties** + +The guarantees of the ordering service (or atomic-broadcast channel) stipulate what happens to a broadcasted message and what relations exist among delivered messages. These guarantees are as follows: + +1. **Safety (consistency guarantees)**: As long as peers are connected for sufficiently long periods of time to the channel (they can disconnect or crash, but will restart and reconnect), they will see an *identical* series of delivered `(seqno, prevhash, blob)` messages. This means the outputs (`deliver()` events) occur in the *same order* on all peers and according to sequence number and carry *identical content* (`blob` and `prevhash`) for the same sequence number. Note this is only a *logical order*, and a `deliver(seqno, prevhash, blob)` on one peer is not required to occur in any real-time relation to `deliver(seqno, prevhash, blob)` that outputs the same message at another peer. Put differently, given a particular `seqno`, *no* two correct peers deliver *different* `prevhash` or `blob` values. Moreover, no value `blob` is delivered unless some client (peer) actually called `broadcast(blob)` and, preferably, every broadcasted blob is only delivered *once*. + + Furthermore, the `deliver()` event contains the cryptographic hash of the data in the previous `deliver()` event (`prevhash`). When the ordering service implements atomic broadcast guarantees, `prevhash` is the cryptographic hash of the parameters from the `deliver()` event with sequence number `seqno-1`. This establishes a hash chain across `deliver()` events, which is used to help verify the integrity of the ordering service output, as discussed in Sections 4 and 5 later. In the special case of the first `deliver()` event, `prevhash` has a default value. + + +1. **Liveness (delivery guarantee)**: Liveness guarantees of the ordering service are specified by a ordering service implementation. The exact guarantees may depend on the network and node fault model. + + In principle, if the submitting client does not fail, the ordering service should guarantee that every correct peer that connects to the ordering service eventually delivers every submitted transaction. + + +To summarize, the ordering service ensures the following properties: + +* *Agreement.* For any two events at correct peers `deliver(seqno, prevhash0, blob0)` and `deliver(seqno, prevhash1, blob1)` with the same `seqno`, `prevhash0==prevhash1` and `blob0==blob1`; +* *Hashchain integrity.* For any two events at correct peers `deliver(seqno-1, prevhash0, blob0)` and `deliver(seqno, prevhash, blob)`, `prevhash = HASH(seqno-1||prevhash0||blob0)`. +* *No skipping*. If an ordering service outputs `deliver(seqno, prevhash, blob)` at a correct peer *p*, such that `seqno>0`, then *p* already delivered an event `deliver(seqno-1, prevhash0, blob0)`. +* *No creation*. Any event `deliver(seqno, prevhash, blob)` at a correct peer must be preceded by a `broadcast(blob)` event at some (possibly distinct) peer; +* *No duplication (optional, yet desirable)*. For any two events `broadcast(blob)` and `broadcast(blob')`, when two events `deliver(seqno0, prevhash0, blob)` and `deliver(seqno1, prevhash1, blob')` occur at correct peers and `blob == blob'`, then `seqno0==seqno1` and `prevhash0==prevhash1`. +* *Liveness*. If a correct client invokes an event `broadcast(blob)` then every correct peer "eventually" issues an event `deliver(*, *, blob)`, where `*` denotes an arbitrary value. + + +## 2. Basic workflow of transaction endorsement + +In the following we outline the high-level request flow for a transaction. + +**Remark:** *Notice that the following protocol _does not_ assume that all transactions are deterministic, i.e., it allows for non-deterministic transactions.* + +### 2.1. The client creates a transaction and sends it to endorsing peers of its choice + +To invoke a transaction, the client sends a `PROPOSE` message to a set of endorsing peers of its choice (possibly not at the same time - see Sections 2.1.2. and 2.3.). The set of endorsing peers for a given `chaincodeID` is made available to client via peer, which in turn knows the set of endorsing peers from endorsement policy (see Section 3). For example, the transaction could be sent to *all* endorsers of a given `chaincodeID`. That said, some endorsers could be offline, others may object and choose not to endorse the transaction. The submitting client tries to satisfy the policy expression with the endorsers available. + +In the following, we first detail `PROPOSE` message format and then discuss possible patterns of interaction between submitting client and endorsers. + +### 2.1.1. `PROPOSE` message format + +The format of a `PROPOSE` message is ``, where `tx` is a mandatory and `anchor` optional argument explained in the following. + +- `tx=`, where + - `clientID` is an ID of the submitting client, + - `chaincodeID` refers to the chaincode to which the transaction pertains, + - `txPayload` is the payload containing the submitted transaction itself, + - `timestamp` is a monotonically increasing (for every new transaction) integer maintained by the client, + - `clientSig` is signature of a client on other fields of `tx`. + + The details of `txPayload` will differ between invoke transactions and deploy transactions (i.e., invoke transactions referring to a deploy-specific system chaincode). For an **invoke transaction**, `txPayload` would consist of two fields + + - `txPayload = `, where + - `operation` denotes the chaincode operation (function) and arguments, + - `metadata` denotes attributes related to the invocation. + + For a **deploy transaction**, `txPayload` would consist of three fields + + - `txPayload = `, where + - `source` denotes the source code of the chaincode, + - `metadata` denotes attributes related to the chaincode and application, + - `policies` contains policies related to the chaincode that are accessible to all peers, such as the endorsement policy. Note that endorsement policies are not supplied with `txPayload` in a `deploy` transaction, but `txPayload of a `deploy` contains endorsement policy ID and its parameters (see Section 3). + +- `anchor` contains _read version dependencies_, or more specifically, key-version pairs (i.e., `anchor` is a subset of `KxN`), that binds or "anchors" the `PROPOSE` request to specified versions of keys in a KVS (see Section 1.2.). If the client specifies the `anchor` argument, an endorser endorses a transaction only upon _read_ version numbers of corresponding keys in its local KVS match `anchor` (see Section 2.2. for more details). + +Cryptographic hash of `tx` is used by all nodes as a unique transaction identifier `tid` (i.e., `tid=HASH(tx)`). +The client stores `tid` in memory and waits for responses from endorsing peers. + +#### 2.1.2. Message patterns + +The client decides on the sequence of interaction with endorsers. For example, a client would typically send `` (i.e., without the `anchor` argument) to a single endorser, which would then produce the version dependencies (`anchor`) which the client can later on use as an argument of its `PROPOSE` message to other endorsers. As another example, the client could directly send `` (without `anchor`) to all endorsers of its choice. Different patterns of communication are possible and client is free to decide on those (see also Section 2.3.). + +### 2.2. The endorsing peer simulates a transaction and produces an endorsement signature + +On reception of a `` message from a client, the endorsing peer `epID` first verifies the client's signature `clientSig` and then simulates a transaction. If the client specifies `anchor` then endorsing peer simulates the transactions only upon read version numbers (i.e., `readset` as defined below) of corresponding keys in its local KVS match those version numbers specified by `anchor`. + +Simulating a transaction involves endorsing peer tentatively *executing* a transaction (`txPayload`), by invoking the chaincode to which the transaction refers (`chaincodeID`) and the copy of the state that the endorsing peer locally holds. + +As a result of the execution, the endorsing peer computes _read version dependencies_ (`readset`) and _state updates_ (`writeset`), also called *MVCC+postimage info* in DB language. + +Recall that the state consists of key/value (k/v) pairs. All k/v entries are versioned, that is, every entry contains ordered version information, which is incremented every time when the value stored under a key is updated. The peer that interprets the transaction records all k/v pairs accessed by the chaincode, either for reading or for writing, but the peer does not yet update its state. More specifically: + +* Given state `s` before an endorsing peer executes a transaction, for every key `k` read by the transaction, pair `(k,s(k).version)` is added to `readset`. +* Additionally, for every key `k` modified by the transaction to the new value `v'`, pair `(k,v')` is added to `writeset`. Alternatively, `v'` could be the delta of the new value to previous value (`s(k).value`). + +If a client specifies `anchor` in the `PROPOSE` message then client specified `anchor` must equal `readset` produced by endorsing peer when simulating the transaction. + +Then, the peer forwards internally `tran-proposal` (and possibly `tx`) to the part of its (peer's) logic that endorses a transaction, referred to as **endorsing logic**. By default, endorsing logic at a peer accepts the `tran-proposal` and simply signs the `tran-proposal`. However, endorsing logic may interpret arbitrary functionality, to, e.g., interact with legacy systems with `tran-proposal` and `tx` as inputs to reach the decision whether to endorse a transaction or not. + +If endorsing logic decides to endorse a transaction, it sends `` message to the submitting client(`tx.clientID`), where: + +- `tran-proposal := (epID,tid,chaincodeID,txContentBlob,readset,writeset)`, + + where `txContentBlob` is chaincode/transaction specific information. The intention is to have `txContentBlob` used as some representation of `tx` (e.g., `txContentBlob=tx.txPayload`). + +- `epSig` is the endorsing peer's signature on `tran-proposal` + +Else, in case the endorsing logic refuses to endorse the transaction, an endorser *may* send a message `(TRANSACTION-INVALID, tid, REJECTED)` to the submitting client. + +Notice that an endorser does not change its state in this step, the updates produced by transaction simulation in the context of endorsement do not affect the state! + +### 2.3. The submitting client collects an endorsement for a transaction and broadcasts it through ordering service + +The submitting client waits until it receives "enough" messages and signatures on `(TRANSACTION-ENDORSED, tid, *, *)` statements to conclude that the transaction proposal is endorsed. As discussed in Section 2.1.2., this may involve one or more round-trips of interaction with endorsers. + +The exact number of "enough" depend on the chaincode endorsement policy (see also Section 3). If the endorsement policy is satisfied, the transaction has been *endorsed*; note that it is not yet committed. The collection of signed `TRANSACTION-ENDORSED` messages from endorsing peers which establish that a transaction is endorsed is called an *endorsement* and denoted by `endorsement`. + +If the submitting client does not manage to collect an endorsement for a transaction proposal, it abandons this transaction with an option to retry later. + +For transaction with a valid endorsement, we now start using the ordering service. The submitting client invokes ordering service using the `broadcast(blob)`, where `blob=endorsement`. If the client does not have capability of invoking ordering service directly, it may proxy its broadcast through some peer of its choice. Such a peer must be trusted by the client not to remove any message from the `endorsement` or otherwise the transaction may be deemed invalid. Notice that, however, a proxy peer may not fabricate a valid `endorsement`. + +### 2.4. The ordering service delivers a transactions to the peers + +When an event `deliver(seqno, prevhash, blob)` occurs and a peer has applied all state updates for blobs with sequence number lower than `seqno`, a peer does the following: + +* It checks that the `blob.endorsement` is valid according to the policy of the chaincode (`blob.tran-proposal.chaincodeID`) to which it refers. + +* In a typical case, it also verifies that the dependencies (`blob.endorsement.tran-proposal.readset`) have not been violated meanwhile. In more complex use cases, `tran-proposal` fields in endorsement may differ and in this case endorsement policy (Section 3) specifies how the state evolves. + +Verification of dependencies can be implemented in different ways, according to a consistency property or "isolation guarantee" that is chosen for the state updates. **Serializability** is a default isolation guarantee, unless chaincode endorsement policy specifies a different one. Serializability can be provided by requiring the version associated with *every* key in the `readset` to be equal to that key's version in the state, and rejecting transactions that do not satisfy this requirement. + +* If all these checks pass, the transaction is deemed *valid* or *committed*. In this case, the peer marks the transaction with 1 in the bitmask of the `PeerLedger`, applies `blob.endorsement.tran-proposal.writeset` to blockchain state (if `tran-proposals` are the same, otherwise endorsement policy logic defines the function that takes `blob.endorsement`). + +* If the endorsement policy verification of `blob.endorsement` fails, the transaction is invalid and the peer marks the transaction with 0 in the bitmask of the `PeerLedger`. It is important to note that invalid transactions do not change the state. + +Note that this is sufficient to have all (correct) peers have the same state after processing a deliver event (block) with a given sequence number. Namely, by the guarantees of the ordering service, all correct peers will receive an identical sequence of `deliver(seqno, prevhash, blob)` events. As the evaluation of the endorsement policy and evaluation of version dependencies in `readset` are deterministic, all correct peers will also come to the same conclusion whether a transaction contained in a blob is valid. Hence, all peers commit and apply the same sequence of transactions and update their state in the same way. + +![Illustration of the transaction flow (common-case path).](http://vukolic.com/hyperledger/flow-4.png) + +Figure 1. Illustration of one possible transaction flow (common-case path). + +--- + +## 3. Endorsement policies + +### 3.1. Endorsement policy specification + +An **endorsement policy**, is a condition on what _endorses_ a transaction. Blockchain peers have a pre-specified set of endorsement policies, which are referenced by a `deploy` transaction that installs specific chaincode. Endorsement policies can be parametrized, and these parameters can be specified by a `deploy` transaction. + +To guarantee blockchain and security properties, the set of endorsement policies **should be a set of proven policies** with limited set of functions in order to ensure bounded execution time (termination), determinism, performance and security guarantees. + +Dynamic addition of endorsement policies (e.g., by `deploy` transaction on chaincode deploy time) is very sensitive in terms of bounded policy evaluation time (termination), determinism, performance and security guarantees. Therefore, dynamic addition of endorsement policies is not allowed, but can be supported in future. + + +### 3.2. Transaction evaluation against endorsement policy + +A transaction is declared valid only if it has been endorsed according to the policy. An invoke transaction for a chaincode will first have to obtain an *endorsement* that satisfies the chaincode's policy or it will not be committed. This takes place through the interaction between the submitting client and endorsing peers as explained in Section 2. + +Formally the endorsement policy is a predicate on the endorsement, and potentially further state that evaluates to TRUE or FALSE. For deploy transactions the endorsement is obtained according to a system-wide policy (for example, from the system chaincode). + +An endorsement policy predicate refers to certain variables. Potentially it may refer to: + +1. keys or identities relating to the chaincode (found in the metadata of the chaincode), for example, a set of endorsers; +2. further metadata of the chaincode; +3. elements of the `endorsement` and `endorsement.tran-proposal`; +4. and potentially more. + +The above list is ordered by increasing expressiveness and complexity, that is, it will be relatively simple to support policies that only refer to keys and identities of nodes. + +**The evaluation of an endorsement policy predicate must be deterministic.** An endorsement shall be evaluated locally by every peer such that a peer does *not* need to interact with other peers, yet all correct peers evaluate the endorsement policy in the same way. + +### 3.3. Example endorsement policies + +The predicate may contain logical expressions and evaluates to TRUE or FALSE. Typically the condition will use digital signatures on the transaction invocation issued by endorsing peers for the chaincode. + +Suppose the chaincode specifies the endorser set `E = {Alice, Bob, Charlie, Dave, Eve, Frank, George}`. Some example policies: + +- A valid signature from on the same `tran-proposal` from all members of E. + +- A valid signature from any single member of E. + +- Valid signatures on the same `tran-proposal` from endorsing peers according to the condition + `(Alice OR Bob) AND (any two of: Charlie, Dave, Eve, Frank, George)`. + +- Valid signatures on the same `tran-proposal` by any 5 out of the 7 endorsers. (More generally, for chaincode with `n > 3f` endorsers, valid signatures by any `2f+1` out of the `n` endorsers, or by any group of *more* than `(n+f)/2` endorsers.) + +- Suppose there is an assignment of "stake" or "weights" to the endorsers, + like `{Alice=49, Bob=15, Charlie=15, Dave=10, Eve=7, Frank=3, George=1}`, + where the total stake is 100: The policy requires valid signatures from a + set that has a majority of the stake (i.e., a group with combined stake + strictly more than 50), such as `{Alice, X}` with any `X` different from + George, or `{everyone together except Alice}`. And so on. + +- The assignment of stake in the previous example condition could be static (fixed in the metadata of the chaincode) or dynamic (e.g., dependent on the state of the chaincode and be modified during the execution). + +- Valid signatures from (Alice OR Bob) on `tran-proposal1` and valid signatures from `(any two of: Charlie, Dave, Eve, Frank, George)` on `tran-proposal2`, where `tran-proposal1` and `tran-proposal2` differ only in their endorsing peers and state updates. + +How useful these policies are will depend on the application, on the desired resilience of the solution against failures or misbehavior of endorsers, and on various other properties. + +## 4 (post-v1). Validated ledger and `PeerLedger` checkpointing (pruning) + +### 4.1. Validated ledger (VLedger) + +To maintain the abstraction of a ledger that contains only valid and committed transactions (that appears in Bitcoin, for example), peers may, in addition to state and Ledger, maintain the *Validated Ledger (or VLedger)*. This is a hash chain derived from the ledger by filtering out invalid transactions. + +The construction of the VLedger blocks (called here *vBlocks*) proceeds as follows. As the `PeerLedger` blocks may contain invalid transactions (i.e., transactions with invalid endorsement or with invalid version dependencies), such transactions are filtered out by peers before a transaction from a block becomes added to a vBlock. Every peer does this by itself (e.g., by using the bitmask associated with `PeerLedger`). A vBlock is defined as a block without the invalid transactions, that have been filtered out. Such vBlocks are inherently dynamic in size and may be empty. An illustration of vBlock construction is given in the figure below. + ![Illustration of the transaction flow (common-case path).](http://vukolic.com/hyperledger/blocks-3.png) + +Figure 2. Illustration of validated ledger block (vBlock) formation from ledger (`PeerLedger`) blocks. + +vBlocks are chained together to a hash chain by every peer. More specifically, every block of a validated ledger contains: + +* The hash of the previous vBlock. + +* vBlock number. + +* An ordered list of all valid transactions committed by the peers since the last vBlock was computed (i.e., list of valid transactions in a corresponding block). + +* The hash of the corresponding block (in `PeerLedger`) from which the current vBlock is derived. + +All this information is concatenated and hashed by a peer, producing the hash of the vBlock in the validated ledger. + + +###4.2. `PeerLedger` Checkpointing + +The ledger contains invalid transactions, which may not necessarily be recorded forever. However, peers cannot simply discard `PeerLedger` blocks and thereby prune `PeerLedger` once they establish the corresponding vBlocks. Namely, in this case, if a new peer joins the network, other peers could not transfer the discarded blocks (pertaining to `PeerLedger`) to the joining peer, nor convince the joining peer of the validity of their vBlocks. + +To facilitate pruning of the `PeerLedger`, this document describes a *checkpointing* mechanism. This mechanism establishes the validity of the vBlocks across the peer network and allows checkpointed vBlocks to replace the discarded `PeerLedger` blocks. This, in turn, reduces storage space, as there is no need to store invalid transactions. It also reduces the work to reconstruct the state for new peers that join the network (as they do not need to establish validity of individual transactions when reconstructing the state by replaying `PeerLedger`, but may simply replay the state updates contained in the validated ledger). + +####4.2.1. Checkpointing protocol + +Checkpointing is performed periodically by the peers every *CHK* blocks, where *CHK* is a configurable parameter. To initiate a checkpoint, the peers broadcast (e.g., gossip) to other peers message ``, where `blockno` is the current blocknumber and `blocknohash` is its respective hash, `stateHash` is the hash of the latest state (produced by e.g., a Merkle hash) upon validation of block `blockno` and `peerSig` is peer's signature on `(CHECKPOINT,blocknohash,blockno,stateHash)`, referring to the validated ledger. + +A peer collects `CHECKPOINT` messages until it obtains enough correctly signed messages with matching `blockno`, `blocknohash` and `stateHash` to establish a *valid checkpoint* (see Section 4.2.2.). + +Upon establishing a valid checkpoint for block number `blockno` with `blocknohash`, a peer: + +* if `blockno>latestValidCheckpoint.blockno`, then a peer assigns `latestValidCheckpoint=(blocknohash,blockno)`, +* stores the set of respective peer signatures that constitute a valid checkpoint into the set `latestValidCheckpointProof`, +* stores the state corresponding to `stateHash` to `latestValidCheckpointedState`, +* (optionally) prunes its `PeerLedger` up to block number `blockno` (inclusive). + +####4.2.2. Valid checkpoints + +Clearly, the checkpointing protocol raises the following questions: *When can a peer prune its `PeerLedger`? How many `CHECKPOINT` messages are "sufficiently many"?*. This is defined by a *checkpoint validity policy*, with (at least) two possible approaches, which may also be combined: + +* *Local (peer-specific) checkpoint validity policy (LCVP).* A local policy at a given peer *p* may specify a set of peers which peer *p* trusts and whose `CHECKPOINT` messages are sufficient to establish a valid checkpoint. For example, LCVP at peer *Alice* may define that *Alice* needs to receive `CHECKPOINT` message from Bob, or from *both* *Charlie* and *Dave*. + +* *Global checkpoint validity policy (GCVP).* A checkpoint validity policy may be specified globally. This is similar to a local peer policy, except that it is stipulated at the system (blockchain) granularity, rather than peer granularity. For instance, GCVP may specify that: + * each peer may trust a checkpoint if confirmed by *11* different peers. + * in a specific deployment in which every orderer is collocated with a peer in the same machine (i.e., trust domain) and where up to *f* orderers may be (Byzantine) faulty, each peer may trust a checkpoint if confirmed by *f+1* different peers collocated with orderers. diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 00000000000..b917907e411 --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,99 @@ +Hyperledger Fabric is a unique implementation of distributed ledger technology +(DLT) that ensures data integrity and consistency while delivering +accountability, transparency, and efficiencies unmatched by other blockchain +or DLT technology. + +Hyperledger Fabric implements a specific type of +[permissioned](glossary.md#permissioned-network) +[blockchain network](glossary.md#blockchain-network) on which members can +track, exchange and interact with digitized assets using +[transactions](glossary.md#transactions) that are governed by smart +contracts - what we call [chaincode](glossary.md#chaincode) - in a secure and +robust manner while enabling [participants](glossary.md#participants) in the +network to interact in a manner that ensures that their transactions and data +can be restricted to an identified subset of network participants - something +we call a [channel](glossary.md#channel). + +The blockchain network supports the ability for members to establish +shared ledgers that contain the source of truth about those digitized +assets, and recorded transactions, that is replicated in a secure manner only +to the set of nodes participating in that channel. + +The Hyperledger Fabric architecture is comprised of the following components: +peer nodes, ordering nodes and the clients applications that are likely +leveraging one of the language-specific Fabric SDKs. These components have +identities derived from certificate authorities. Hyperledger Fabric also +offers a certificate authority service, *fabric-ca* but, you may substitute +that with your own. + +All peer nodes maintain the ledger/state by committing +transactions. In that role, the peer is called a +[committer](glossary.md#committer). Some peers are also responsible for +simulating transactions by executing chaincodes (smart contracts) and endorsing +the result. In that role the peer is called an [endorser](glossary.md#endorser). +A peer may be an endorser for certain types of transactions and just a ledger +maintainer (committer) for others. + +The [orderers](glossary.md#orderer) consent on the order of transactions in a +block to be committed to the ledger. In common blockchain architectures +(including earlier versions of the Hyperledger Fabric) the roles played by +the peer and orderer nodes were unified (cf. validating peer in Hyperledger +Fabric v0.6). The orderers also play a fundamental role in the creation and +management of channels. + +Two or more [participants](glossary.md#participant) may create and join a +channel, and begin to interact. Among other things, the policies governing the +channel membership and chaincode lifecycle are specified at the time of +channel creation. Initially, the members in a channel agree on +the terms of the chaincode that will govern the transactions. When consensus +is reached on the [proposal](glossary.md#proposal) to deploy a given chaincode +(as governed by the life cycle policy for the channel), it is committed to +the ledger. + +Once the chaincode is deployed to the peer nodes in the channel, +[end users](glossary.md#end-users) with the right privileges can propose +transactions on the channel by using one of the language-specific client SDKs +to invoke functions on the deployed chaincode. + +The proposed transactions are sent to endorsers that execute the chaincode +(also called "simulated the transaction"). On successful execution, endorse +the result using the peer's identity and return the result to the client that +initiated the proposal. + +The client application ensures that the results from the endorsers are +consistent and signed by the appropriate endorsers, according to the endorsement +policy for that chaincode and, if so, the application then sends the transaction, +comprised of the result and endorsements, to the ordering service. + +Ordering nodes order the transactions - the result and endorsements received +from the clients - into a block which is then sent to the peer nodes to be +committed to the ledger. The peers then validate the transaction using the +endorsement policy for the transaction's chaincode and against the ledger for +consistency of result. + +Some key capabilities of Hyperledger Fabric include: + +- Allows for complex query for applications that need ability to handle complex +data structures. + +- Implements a permissioned network, also known as a consortia network, where +all members are known to each other. + +- Incorporates a modular approach to various capabilities, enabling network +designers to plug in their preferred implementations for various capabilities +such as consensus (ordering), identity management, and encryption. + +- Provides a flexible approach for specifying policies and pluggable mechanisms +to enforce them. + +- Ability to have multiple channels, isolated from one another, that +allows for multi-lateral transactions amongst select peer nodes, thereby +ensuring high degrees of privacy and confidentiality required by competing +businesses and highly regulated industries on a common network. + +- Network scalability and performance are achieved through separation of +chaincode execution from transaction ordering, which limits the required levels +of trust and verification across nodes for optimization. + +For a deeper dive into the details, please visit +[this document](arch-deep-dive.md). diff --git a/docs/glossary.md b/docs/glossary.md index 45006b1a58a..3400ee67bfa 100644 --- a/docs/glossary.md +++ b/docs/glossary.md @@ -1,35 +1,41 @@ # Hyperledger Fabric Glossary *Note: This glossary is structured to prioritize new terms and features specific -to v1.0 architecture.  It makes the assumption that one already possesses a +to architecture.  It makes the assumption that one already possesses a working familiarity with the basic tenets of blockchain.* -__Blockchain Network__ – A blockchain network consists of, at minimum, one peer +## Blockchain Network +A blockchain network consists of, at minimum, one peer (responsible for endorsing and committing transactions) leveraging an ordering service, and a membership services component (certificate authority) that distributes and revokes cryptographic certificates representative of user identities and permissions. -__Permissioned Network__ - A blockchain network where any entity (node) is +## Permissioned Network +A blockchain network where any entity (node) is required to maintain a member identity on the network. End users must be authorized and authenticated in order to use the network. -__Peer__ - Component that executes, and maintains a ledger of, transactions.   +## Peer +Peer is a component that executes, and maintains a ledger of, transactions.   There are two roles for a peer – endorser and committer.  The architecture has been designed such that a peer is always a committer, but not necessarily always an endorser. Peers play no role in the ordering of transactions. -__Member__ – A Member is a participant (such as a company or organization) that +## Member +A Member is a participant (such as a company or organization) that operates components - Peers, Orderers, and applications - in the blockchain network.  A member is identified by its CA certificate (i.e. a unique enrollment).   A Member’s peer will be leveraged by end users in order to perform transaction operations on specific channels. -__Transaction__ - Refers to an operation in which an authorized end user +## Transaction +Refers to an operation in which an authorized end user performs read/write operations against the ledger. There are three unique types of transactions - deploy, invoke, and query. -__End User__ – An end user is someone who would interact with the blockchain +## End User +An end user is someone who would interact with the blockchain through a set of published APIs (i.e. the hfc SDK).  You can have an admin user who will typically grant permissions to the Member’s components, and a client user, who, upon proper authentication through the admin user, will drive @@ -37,44 +43,51 @@ chaincode applications (deploy, invoke, query) on various channels.  In the cas of self-executing transactions, the application itself can also be thought of as the end user. -__Ordering Service__ - A centralized or decentralized service that orders +## Ordering Service +A centralized or decentralized service that orders transactions in a block.  You can select different implementations of the "ordering" function - e.g "solo" for simplicity and testing, Kafka for crash fault tolerance, or sBFT/PBFT for byzantine fault tolerance. You can also develop your own protocol to plug into the service. -__Consensus__ - A broader term overarching the entire transactional flow, which +## Consensus +A broader term overarching the entire transactional flow, which serves to generate an agreement on the order and to confirm the correctness of the set of transactions constituting a block. -__Orderer__ - One of the network entities that form the ordering service. A +## Orderer +One of the network entities that form the ordering service. A collection of ordering service nodes (OSNs) will order transactions into blocks according to the network's chosen ordering implementation. In the case of "solo", only one OSN is required. Transactions are "broadcast" to orderers, and then "delivered" as blocks to the appropriate channel. -__Endorser__ – A specific peer role, where the Endorser peer is responsible for +## Endorser +A specific peer role, where the Endorser peer is responsible for simulating transactions, and in turn preventing unstable or non-deterministic transactions from passing through the network.  A transaction is sent to an endorser in the form of a transaction proposal.  All endorsing peers are also committing peers (i.e. they write to the ledger). -__Committer__ –  A specific peer role, where the Committing peer appends the +## Committer +A specific peer role, where the Committing peer appends the validated transactions to the channel-specific ledger.  A peer can act as both an endorser and committer, but in more regulated circumstances might only serve as a committer. -__Bootstrap__ – The initial setup of a network.  There is the bootstrap of a +## Bootstrap +The initial setup of a network.  There is the bootstrap of a peer network, during which policies, system chaincodes, and cryptographic materials (certs) are disseminated amongst participants, and the bootstrap of an ordering network.  The bootstrap of the ordering network must precede the bootstrap of the peer network, as a peer network is contingent upon the presence of an ordering service. A network need only be “bootstrapped” once. -__Block__ - A batch of ordered transactions, potentially containing ones of an +## Block +A batch of ordered transactions, potentially containing ones of an invalid nature, that is delivered to the peers for validation and committal. -__System chain__ - Necessary in order to initialize a blockchain network. +## System chain Contains a configuration block defining the network at a system level. The system chain lives within the ordering service, and similar to a channel, has an initial configuration containing information @@ -88,20 +101,23 @@ of channels. For instance, a collection of financial institutions may form a consortium (represented through the system chain), and then proceed to create channels relative to their aligned and varying business agendas. -__Channel__ - Formed as an offshoot of the system chain; and best thought of +## Channel +A Channel is formed as an offshoot of the system chain; and best thought of as a “topic” for peers to subscribe to, or rather, a subset of a broader blockchain network. A peer may subscribe on various channels and can only access the transactions on the subscribed channels.  Each channel will have a unique ledger, thus accommodating confidentiality and execution of multilateral contracts. -__Multi-channel__ - The fabric will allow for multiple channels with a +## Multi-channel +The fabric will allow for multiple channels with a designated ledger per channel.  This capability allows for multilateral contracts where only the restricted participants on the channel will submit, endorse, order, or commit transactions on that channel.  As such, a single peer can maintain multiple ledgers without compromising privacy and confidentiality. -__Configuration Block__ - Contains the configuration data defining members and +## Configuration Block +Contains the configuration data defining members and policies for a system chain or channel(s). Any changes to the channel(s) or overall network (e.g. a new member successfully joining) will result in a new configuration block being appended to the appropriate chain.  This block will @@ -109,10 +125,12 @@ contain the contents of the genesis block, plus the delta. The policy to alter or edit a channel-level configuration block is defined through the Configuration System Chaincode (CSCC). -__Genesis Block__ – The configuration block that initializes a blockchain +## Genesis Block +The configuration block that initializes a blockchain network or channel, and also serves as the first block on a chain. -__Ledger__ – An append-only transaction log managed by peers.  Ledger keeps the +## Ledger +An append-only transaction log managed by peers.  Ledger keeps the log of ordered transaction batches. There are two denotations for ledger; peer and validated. The peer ledger contains all batched transactions coming out of the ordering service, some of which may in fact be invalid. The validated ledger @@ -120,56 +138,66 @@ will contain fully endorsed and validated transaction blocks. In other words, transactions in the validated ledger have passed the entire gamut of "consensus" - i.e. they have been endorsed, ordered, and validated. -__Dynamic membership__ - The fabric will allow for endorsers and committers to +## Dynamic membership +he fabric will allow for endorsers and committers to come and go based on membership, and the blockchain network will continue to operate. Dynamic membership is critical when businesses grow and members need to be added or removed for various reasons. -__Query/Non-Key Value Query__ – using couchDB 2.0 you now have the capability +## Query/Non-Key Value Query +using couchDB 2.0 you now have the capability to leverage an API to perform more complex queries against combinations of variables, including time ranges, transaction types, users, etc.  This feature allows for auditors and regulators to aggregate and mine large chunks of data. -__Gossip Protocol__ – communication protocol used among peers in a channel, to +## Gossip Protocol +A communication protocol used among peers in a channel, to maintain their network and to elect Leaders, through which funnels all communications with the Ordering Service. Gossip allows for data dissemination, therein providing support for scalability due to the fact that not all peers are required to execute transactions and communicate with the ordering service. -__System Chaincode (SCC)__ - System Chaincode is a chaincode built with the peer +## System Chaincode +System Chaincode (SCC) is a chaincode built with the peer and run in the same process as the peer. SCC is responsible for broader configurations of fabric behavior, such as timing and naming services. -__Lifecycle System Chaincode (LSCC)__ - Handles deployment, upgrade and -termination transactions for user chaincodes. +## Lifecycle System Chaincode +Lifecycle System Chaincode (LSCC) is a system chaincode that handles deployment, +upgrade and termination transactions for user chaincodes. -__Configuration System Chaincode (CSCC)__ - A “management” system chaincode that +## Configuration System Chaincode +Configuration System Chaincode (CSCC) is a “management” system chaincode that handles configuration requests to alter an aspect of a channel (e.g. add a new member).  The CSCC will interrogate the channel’s policies to determine if a new configuration block can be created. -__Endorsement System Chaincode (ESCC)__ - Handles the endorsement policy for -specific pieces of chaincode deployed on a network, and defines the necessary -parameters (percentage or combination of signatures from endorsing peers) for a -transaction proposal to receive a successful proposal response -(i.e. endorsement).  Deployments and invocations of user chaincodes both require -a corresponding ESCC, which is defined at the time of the deployment transaction -proposal for the user chaincode. - -__Validation System Chaincode (VSCC)__ - Handles the validation policy for +## Endorsement System Chaincode +Endorsement System Chaincode (ESCC) is a system chaincode that andles the +endorsement policy for specific pieces of chaincode deployed on a network, +and defines the necessary parameters (percentage or combination of signatures +from endorsing peers) for a transaction proposal to receive a successful +proposal response (i.e. endorsement).  Deployments and invocations of user +chaincodes both require a corresponding ESCC, which is defined at the time of +the deployment transaction proposal for the user chaincode. + +## Validation System Chaincode +Validation System Chaincode (VSCC) Handles the validation policy for specific pieces of chaincode deployed on a network.  Deployments and invocations of user chaincodes both require a corresponding VSCC, which is defined at the time of the deployment transaction proposal for the user chaincode. VSCC validates the specified level of "endorsement" (i.e. endorsement policy) in order to prevent malicious or faulty behavior from the client. -__Policy__ – There are policies for endorsement, validation, block committal, +## Policy +There are policies for endorsement, validation, block committal, chaincode management and network/channel management.  Policies are defined through system chaincodes, and contain the requisite specifications for a network action to succeed.  For example, an endorsement policy may require that 100% of endorsers achieve the same result upon transaction simulation. -__Endorsement policy__ - A blockchain network must establish rules that govern +## Endorsement policy +A blockchain network must establish rules that govern the endorsement (or not) of proposed, simulated transactions. This endorsement policy could require that a transaction be endorsed by a minimum number of endorsing peers, a minimum percentage of endorsing peers, or by all endorsing @@ -178,20 +206,24 @@ curated based on the application and the desired level of resilience against misbehavior (deliberate or not) by the endorsing peers. A distinct endorsement policy for deploy transactions, which install new chaincode, is also required. -__Proposal__ - a transaction request sent from a client or admin user to one or +## Proposal +A transaction request sent from a client or admin user to one or more peers in a network; examples include deploy, invoke, query, or configuration request. -__Deploy__ – refers to the function through which chaincode applications are +## Deploy +Refers to the function through which chaincode applications are deployed on `chain`. A deploy is first sent from the client SDK or CLI to a Lifecycle System Chaincode in the form of a proposal. -__Invoke__ – Used to call chaincode functions.  Invocations are captured as +## Invoke +Used to call chaincode functions.  Invocations are captured as transaction proposals, which then pass through a modular flow of endorsement, ordering, validation, committal.  The structure of invoke is a function and an array of arguments. -__Membership Services__ - Membership Services manages user identities on a +## Membership Services +Membership Services manages user identities on a permissioned blockchain network; this function is implemented through the `fabric-ca` component.  `fabric-ca` is comprised of a client and server, and handles the distribution and revocation of enrollment materials (certificates), @@ -213,7 +245,8 @@ certificates persist on the blockchain, and enable authorized auditors to associate, and identify the transacting parties for otherwise un-linkable transactions. -__Membership Service Provider (MSP)__ - Refers to an abstract component of the +## Membership Service Provider +The Membership Service Provider (MSP) refers to an abstract component of the system that provides (anonymous) credentials to clients, and peers for them to participate in a Hyperledger/fabric network. Clients use these credentials to authenticate their transactions, and peers use these credentials to authenticate @@ -223,29 +256,34 @@ membership services components defined, in such a way that alternate implementations of this can be smoothly plugged in without modifying the core of transaction processing components of the system. -__Initialize__ – a chaincode method to define the assets and parameters in a +## Initialize +A chaincode method to define the assets and parameters in a piece of chaincode prior to issuing deploys and invocations.  As the name implies, this function should be used to do any initialization to the chaincode, such as configure the initial state of a key/value pair on the ledger. -__appshim__ - An application client used by ordering service nodes to +## appshim +An application client used by ordering service nodes to process "broadcast" messages arriving from clients or peers. This shim allows the ordering service to perform membership-related functionality checks. In other words, is a peer or client properly authorized to perform the requested function (e.g. upgrade chaincode or reconfigure channel settings). -__osshim__ - An ordering service client used by the application to process +## osshim +An ordering service client used by the application to process ordering service messages (i.e. "deliver" messages) that are advertised within a channel. -__Hyperledger Fabric Client Software Development Kit (SDK)__ – Provides a -powerful class of APIs and contains myriad “methods” or “calls” that expose the -capabilities and functionalities in the Hyperledger Fabric code base.  For -example, `addMember`, `removeMember`. The Fabric SDK comes in three flavors - -Node.js, Java, and Python - allowing developers to write application code in -any of these programming languages. +## Hyperledger Fabric Client SDK +Provides a powerful set of APIs and contains myriad “methods” or “calls” +that expose the capabilities and functionalities in the Hyperledger Fabric +code base.  For example, `addMember`, `removeMember`. The Fabric SDK +comes in multiple flavors - Node.js, Java, and Python, for starters - +thus, allowing developers to write application code in any of those +programming languages. -__Chaincode__ – Embedded logic that encodes the rules for specific types of +## Chaincode +Embedded logic that encodes the rules for specific types of network transactions. Developers write chaincode applications, which are then deployed onto a chain by an appropriately authorized member. End users then invoke chaincode through a client-side application that interfaces with a diff --git a/docs/index.md b/docs/index.md index c885a177f36..75a4fc6fba7 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,10 +1,3 @@ -# Incubation Notice - -This project is a Hyperledger project in _Incubation_. It was proposed to the -community and documented [here](https://goo.gl/RYQZ5N). Information on what -_Incubation_ entails can be found in the [Hyperledger Project Lifecycle -document](https://goo.gl/4edNRc). - [![Build Status](https://jenkins.hyperledger.org/buildStatus/icon?job=fabric-merge-x86_64)](https://jenkins.hyperledger.org/view/fabric/job/fabric-merge-x86_64/) [![Go Report Card](https://goreportcard.com/badge/github.com/hyperledger/fabric)](https://goreportcard.com/report/github.com/hyperledger/fabric) [![GoDoc](https://godoc.org/github.com/hyperledger/fabric?status.svg)](https://godoc.org/github.com/hyperledger/fabric) @@ -12,102 +5,47 @@ document](https://goo.gl/4edNRc). # Hyperledger Fabric -The Fabric is an implementation of blockchain technology, leveraging familiar -and proven technologies. It is a modular architecture allowing pluggable -implementations of various function. It features powerful container technology -to host any mainstream language for smart contracts development. +The Hyperledger Fabric is an implementation of blockchain technology, +leveraging familiar and proven technologies. It offers a modular +[architecture](architecture.md) allowing pluggable implementations of +various capabilities, such as smart contracts (chaincode), cryptography, +consensus, ledger datastore and more. It leverages powerful container technology +for its deployment and for providing the execution environment for the smart +contracts (chaincode) such that the network is protected from malicious code. -## Releases +## Get Hyperledger Fabric The Fabric releases are documented [here](releases.md). We -released our second release under the governance of the Hyperledger Project - -v0.6-preview in October, 2016. +published our second release under the governance of the Hyperledger Project - +[v0.6-preview](http://hyperledger-fabric.readthedocs.io/en/v0.6/) in October, +2016. -If you are seeking a stable version of the Hyperledger Fabric on which to +If you are seeking a **stable** version of the Hyperledger Fabric on which to develop applications or explore the technology, we **strongly recommend** that -you use the v0.6 [Starter Kit](http://hyperledger-fabric.readthedocs.io/en/v0.6/starter/fabric-starter-kit/) -while the v1.0 architectural refactoring is in development. - -If you'd like a taste of what the Fabric v1.0 architecture has in store, we -invite you to read the preview [here](abstract_v1.md). Finally, if you are -adventurous, we invite you to explore the current state of development with -the caveat that it is not yet completely stable. - -## Contributing to the project - -We welcome contributions to the Hyperledger Project in many forms. There's -always plenty to do! Full details of how to contribute to this project are -documented in the [Fabric developer's guide](#fabric-developer-guide) below. - -## Maintainers +you use the +[v0.6 Starter Kit](http://hyperledger-fabric.readthedocs.io/en/v0.6/starter/fabric-starter-kit/) +while the architectural refactoring for our upcoming v1.0 release is still in +development. -The project's [maintainers](MAINTAINERS.md) are responsible for reviewing and -merging all patches submitted for review and they guide the over-all technical -direction of the project within the guidelines established by the Hyperledger -Project's Technical Steering Committee (TSC). +However, if you'd like a taste of what the Hyperledger Fabric architecture has +in store, we invite you to read the preview [here](abstract_v1.md). -## Communication - -We use [Hyperledger chat](https://chat.hyperledger.org/) for communication and -Google Hangouts™ for screen sharing between developers. Our development -planning and prioritization is done in [JIRA](https://jira.hyperledger.org), -and we take longer running discussions/decisions to the -[mailing list](http://lists.hyperledger.org/mailman/listinfo/hyperledger-fabric). - -## Still Have Questions? -We try to maintain a comprehensive set of documentation (see below) for various -audiences. However, we realize that often there are questions that remain -unanswered. For any technical questions relating to the Hyperledger Fabric -project not answered in this documentation, please use -[StackOverflow](http://stackoverflow.com/questions/tagged/hyperledger). If you -need help finding things, please don't hesitate to send a note to the -[mailing list](http://lists.hyperledger.org/mailman/listinfo/hyperledger-fabric), -or ask on [chat]((https://chat.hyperledger.org/)). +If you are *adventurous*, we invite you to explore the current state of +development using the [v1.0 Starter Kit](gettingstarted.md), with the caveat +that it is not yet completely stable. We hope to have a stable v1.0-alpha +release in the very near term. # Hyperledger Fabric Documentation -The Hyperledger Fabric is an implementation of blockchain technology, that has -been collaboratively developed under the Linux Foundation's -[Hyperledger Project](http://hyperledger.org). It leverages familiar and -proven technologies, and offers a modular architecture -that allows pluggable implementations of various function including membership -services, consensus, and smart contracts (Chaincode) execution. It features -powerful container technology to host any mainstream language for smart -contracts development. - -## Table of Contents - -Below, you'll find the following sections: - -* [Read All About It](#read-all-about-it) -* [Developer guides](#developer-guides) - * [Application developer's guide](#application-developer-guide) - * [Fabric developer's guide](#fabric-developer-guide) -* [Operations guide](#operations-guide) - -## Read all about it - If you are new to the project, you might want to familiarize yourself with some of the basics before diving into either developing applications using the Hyperledger Fabric, or collaborating with us on our journey to continuously extend and improve its capabilities. -- [Canonical use cases](biz/usecases.md) +- [Use cases](biz/usecases.md) - [Glossary](glossary.md): to understand the terminology that we use throughout -the Fabric project's documentation. -- [Fabric FAQs](https://github.com/hyperledger/fabric/tree/master/docs/FAQ) - -# Developer guides - -There are two distinct types of developers for which we have authored this -documentation: 1) [application developers](#application-developer-guide) -building applications and solutions using the Hyperledger Fabric 2) developers -who want to engage in the [development of the Hyperledger Fabric](#fabric-developer-guide) -itself. We distinguish these two personas because the development setup -for the Hyperledger Fabric developer is much more involved as they need to -be able to build the software and there are additional prerequisites that need -to be installed that are largely unnecessary for developers building -applications. +the Hyperledger Fabric project's documentation. +- [Hyperledger Fabric FAQ](https://github.com/hyperledger/fabric/tree/master/docs/FAQ) ## Application developer guide @@ -121,30 +59,38 @@ a step-by-step guide to developing and testing Chaincode. - [Chaincode FAQ](FAQ/chaincode_FAQ.md): a FAQ for all of your burning questions relating to Chaincode. -## Fabric developer guide - -- [Making code contributions](CONTRIBUTING.md): First, you'll want to familiarize - yourself with the project's contribution guidelines. -- [Setting up the development environment](dev-setup/devenv.md): after that, you - will want to set up your development environment. -- [Building the Fabric core](dev-setup/build.md): next, try building the project - in your local development environment to ensure that everything is set up - correctly. -- [Building outside of Vagrant](dev-setup/build.md#building-outside-of-vagrant): - for the *adventurous*, you might try to build outside of the standard - Vagrant development environment. -- [Logging control](Setup/logging-control.md): describes how to tweak the logging - levels of various components within the Fabric. -- [License header](dev-setup/headers.txt): every source file must include this - license header modified to include a copyright statement for the principle - author(s). - # Operations guide (coming soon) **Note:** if you are looking for instructions to operate the fabric for a POC, -we recommend that you use the more stable v0.6 [Starter Kit](http://hyperledger-fabric.readthedocs.io/en/v0.6/starter/fabric-starter-kit/) +we recommend that you use the more stable [v0.6 Starter Kit](http://hyperledger-fabric.readthedocs.io/en/v0.6/starter/fabric-starter-kit/). +However, the [v1.0 Starter Kit](gettingstarted.md) is also available, though +subject to change as we stabilize the code for the upcoming v1.0-alpha release. + +## Contributing to the project + +We welcome contributions to the Hyperledger Project in many forms. There's +always plenty to do! Full details of how to contribute to this project are +documented in the [Contribution Guidelines](CONTRIBUTING.md). + +## Still Have Questions? +We try to maintain a comprehensive set of documentation (see below) for various +audiences. However, we realize that often there are questions that remain +unanswered. For any technical questions relating to the Hyperledger Fabric +project not answered in this documentation, please use +[StackOverflow](http://stackoverflow.com/questions/tagged/hyperledger). If you +need help finding things, please don't hesitate to send a note to the +[mailing list](http://lists.hyperledger.org/mailman/listinfo/hyperledger-fabric), +or ask on [RocketChat]((https://chat.hyperledger.org/)) (an alternative to +Slack). + +# Incubation Notice + +This project is a Hyperledger project in _Incubation_. It was proposed to the +community and documented [here](https://goo.gl/RYQZ5N). Information on what +_Incubation_ entails can be found in the [Hyperledger Project Lifecycle +document](https://goo.gl/4edNRc). # License The Hyperledger Project uses the [Apache License Version 2.0](LICENSE) software diff --git a/docs/whyfabric.md b/docs/whyfabric.md index 4a93771ebc7..22eb0b4488c 100644 --- a/docs/whyfabric.md +++ b/docs/whyfabric.md @@ -1,40 +1,48 @@ -# Why Fabric? +# Why Hyperledger Fabric? -Hyperledger Fabric (hereafter referred to as Fabric) is a blockchain platform -designed to allow the exchange of an asset or the state of an asset to be consented -upon, maintained, and viewed by all parties in a permissioned group. A key characteristic -of Fabric is that the asset is defined digitally, with all participants simply agreeing -on its representation/characterization. As such, Fabric can support a broad range of -asset types; ranging from the tangible (real estate & hardware) to the intangible -(contracts and IP). +The Hyperledger Fabric project is delivering a blockchain platform +designed to allow the exchange of an asset or the state of an asset to be +consented upon, maintained, and viewed by all parties in a permissioned group. +A key characteristic of Hyperledger Fabric is that the asset is defined +digitally, with all participants simply agreeing on its +representation/characterization. As such, Hypewrledger Fabric can support a +broad range of asset types; ranging from the tangible (real estate and hardware) +to the intangible (contracts and intellectual property). -The technology is based on a standard blockchain concept - a shared, replicated ledger. -However, Fabric is based on a permissioned network, meaning all participants are -required to be authenticated in order to participate and transact on -the blockchain. Moreover, these identities can be used to govern certain levels of -access control (e.g. this user can read the ledger, but cannot exchange or transfer assets). This -dependence on identity is a great advantage in that varying consensus algorithms (e.g. -BFT based or Kafka) can be implemented in place of the more compute-intensive Proof-of-Work and -Proof-of-Stake varieties. As a result, permissioned networks tend to provide higher -throughput and performance. +The technology is based on a standard blockchain concept - a shared, replicated +ledger. However, Hyperledger Fabric is based on a +[permissioned network](glossary.md#permissioned-network), meaning all +participants are required to be authenticated in order to participate and +transact on the blockchain. Moreover, these identities can be used to govern +certain levels of access control (e.g. this user can read the ledger, but cannot +exchange or transfer assets). This dependence on identity is a great advantage +in that varying consensus algorithms (e.g. byzantine or crash fault tolerant) +can be implemented in place of the more compute-intensive Proof-of-Work and +Proof-of-Stake varieties. As a result, permissioned networks tend to provide +higher transaction throughput rates and performance. -Once an organization is granted access to the blockchain network, it then has the ability -to create and maintain a private channel with other specified members. For example, -let's assume there are four organizations trading jewels. They may decide to use -Fabric because they trust each other, but not to an unconditional extent. They can -all agree on the business logic for trading the jewels, and can all maintain a global -ledger to view the current state of their jewel market (call this the consortium channel). -Additionally, two or more of these organizations might decide to form an alternate -private blockchain for a certain exchange that they want to keep confidential -(e.g. price X for quantity Y of asset Z). They can perform this trade without affecting -their broader consortium channel, or, if desired, this private channel can broadcast some -level of reference data to their consortium channel. +Once an organization is granted access to the +[blockchain network](glossary.md#blockchain-network), it then has the ability +to create and maintain a private [channel](glossary.md#channel) with other +specified members. For example, let's assume there are four organizations +trading jewels. They may decide to use Hyperledger Fabric because they trust +each other, but not to an unconditional extent. They can all agree on the +business logic for trading the jewels, and can all maintain a global ledger to +view the current state of their jewel market (call this the consortium channel). +Additionally, two or more of these organizations might decide to form an +alternate private blockchain for a certain exchange that they want to keep +confidential (e.g. price X for quantity Y of asset Z). They can perform this +trade without affecting their broader consortium channel, or, if desired, +this private channel can broadcast some level of reference data to their +consortium channel. -This is powerful! This provides for great flexibility and potent capabilities, along with -the interoperability of multiple blockchain ledgers within one consortium. This is the -first of its kind and allows organizations to curate Fabric to support the myriad use -cases for different businesses and industries. Hyperledger Fabric has already been +This is powerful! This provides for great flexibility and potent capabilities, +along with the interoperability of multiple blockchain ledgers within one +consortium. This is the first of its kind and allows organizations to curate +Hyperledger Fabric to support the myriad use cases for different businesses +and industries. Hyperledger Fabric has already been successfully implemented in the banking, finance, and retail industries. -We welcome you to the Hyperledger Fabric community and are glad to hear your architectural -and business needs, and help determine how Fabric can be leveraged to support your use case. +We welcome you to the Hyperledger Fabric community and are keen to learn of your +architectural and business requirements, and help determine how Hyperledger +Fabric can be leveraged to support your use cases. diff --git a/mkdocs.yml b/mkdocs.yml index 6aa3589bc4e..09f493a0140 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -9,9 +9,10 @@ theme_dir: 'docs/custom_theme' pages: - Home: index.md - Why Hyperledger Fabric?: whyfabric.md -- Fabric overview: overview.md +- Fabric Overview: overview.md +- Architecture Overview: architecture.md +- Architecture Deep Dive: arch-deep-dive.md #- Fabric capabilities: capabilities.md -#- Fabric architecture: architecture.md - Getting started: gettingstarted.md #- Chaincode and sample program: chaincode.md - Glossary: glossary.md