From a263d4d69c8406c33206a61bac657174ca73bac6 Mon Sep 17 00:00:00 2001 From: Josh Horton Date: Thu, 2 Dec 2021 12:01:00 -0500 Subject: [PATCH] Final peer for gateway (#3091) * Peers for Gateway Signed-off-by: Josh Horton * latest 11-29 review updates Signed-off-by: Josh Horton * Final doc for peers topic Signed-off-by: Josh Horton * final doc for peers gateway Signed-off-by: Josh Horton * Andy final review Signed-off-by: Josh Horton (cherry picked from commit 516bc8bab548a484e2e62fee24b0f4fed5bf6873) --- docs/source/peers/peers.md | 703 +++++++++++++++++-------------------- 1 file changed, 317 insertions(+), 386 deletions(-) diff --git a/docs/source/peers/peers.md b/docs/source/peers/peers.md index 6437ad7ff9d..87cc9976f44 100644 --- a/docs/source/peers/peers.md +++ b/docs/source/peers/peers.md @@ -1,193 +1,200 @@ # Peers -A blockchain network is comprised primarily of a set of *peer nodes* (or, simply, *peers*). -Peers are a fundamental element of the network because they host ledgers and smart -contracts. Recall that a ledger immutably records all the transactions generated -by smart contracts (which in Hyperledger Fabric are contained in a *chaincode*, -more on this later). Smart contracts and ledgers are used to encapsulate the -shared *processes* and shared *information* in a network, respectively. These -aspects of a peer make them a good starting point to understand a Fabric network. - -Other elements of the blockchain network are of course important: ledgers and -smart contracts, orderers, policies, channels, applications, organizations, -identities, and membership, and you can read more about them in their own -dedicated sections. This section focusses on peers, and their relationship to those -other elements in a Fabric network. +A fundamental element of a Hyperledger Fabric blockchain network is the set of +*peer nodes* (or, simply, *peers*). Peers are fundamental because they manage +ledgers and smart contracts. Starting in Hyperledger Fabric v2.4, peers also +manage transaction proposals and endorsements by running the +[Fabric Gateway](../gateway.html) service. Recall that a ledger +immutably records all of the transactions generated by smart contracts (which in + Hyperledger Fabric are contained +in *chaincode*, more on this later) and endorsed by the required organizations. +Smart contracts and ledgers encapsulate the *processes* and *information*, respectively, +that are shared by channel peers. These aspects of peers make them a good starting +point for understanding a Fabric network. + +Besides peers, other elements of a Fabric blockchain network are also important: +ledgers and smart contracts, orderers, policies, channels, client applications, +organizations, identities, and membership, which you can read about in their own dedicated sections. +This section focuses on peers, and their relationships to these other elements in a Fabric +network. ![Peer1](./peers.diagram.1.png) -*A blockchain network is comprised of peer nodes, each of which can hold copies -of ledgers and copies of smart contracts. In this example, the network N -consists of peers P1, P2 and P3, each of which maintain their own instance of -the distributed ledger L1. P1, P2 and P3 use the same chaincode, S1, to access -their copy of that distributed ledger*. +*A Fabric blockchain network (above) is comprised of peers (non-ordering nodes), each +of which stores and manages copies of ledgers and smart contracts. In this example, the Fabric +network N consists of peers P1, P2 and P3, each of which maintains its own instance of +the distributed ledger L1. P1, P2 and P3 each invoke the same chaincode, S1, +to access their respective copies of the distributed ledger.* -Peers can be created, started, stopped, reconfigured, and even deleted. They -expose a set of APIs that enable administrators and applications to interact -with the services that they provide. We'll learn more about these services in -this section. +Peers are a flexible and redundant element that can be created, started, +stopped, reconfigured and deleted. Peers expose a set of APIs that enable +client applications to interact with the services that peers provide --- the +[Fabric Gateway](../gateway.html) service in particular. -### A word on terminology +## Chaincode terminology -Fabric implements **smart contracts** with a technology concept it calls -**chaincode** --- simply a piece of code that accesses the ledger, written in -one of the supported programming languages. In this topic, we'll usually use the -term **chaincode**, but feel free to read it as **smart contract** if you're -more used to that term. It's the same thing! If you want to learn more about -chaincode and smart contracts, check out our [documentation on smart contracts -and chaincode](../smartcontract/smartcontract.html). +Fabric implements **smart contracts** through a type of logic called **chaincode** +--- code that accesses the ledger --- which is written +using the [chaincode APIs](../sdk_chaincode.html). In this topic, we use the term +**chaincode**, but feel free to interpret chaincode as a **smart contract** if +that concept is more familiar. To learn more about chaincode and smart contracts, check out +the [documentation on smart contracts and chaincode](../smartcontract/smartcontract.html). ## Ledgers and Chaincode -Let's look at a peer in a little more detail. We can see that it's the peer that -hosts both the ledger and chaincode. More accurately, the peer actually hosts -*instances* of the ledger, and *instances* of chaincode. Note that this provides -a deliberate redundancy in a Fabric network --- it avoids single points of -failure. We'll learn more about the distributed and decentralized nature of a -blockchain network later in this section. +Now let's look at a peer in a little more detail. We can see that it's the +peer that hosts both the ledger and chaincode, in addition to services such as +Fabric Gateway. More precisely, the peer hosts *instances* of the ledger, +and *instances* of chaincode, because blockchain requires consistent replicas of +data and smart contracts across peers in a channel. This design provides a deliberate +redundancy in a Fabric network --- avoiding single points of failure and providing +consistent, current ledgers. We'll learn more about the distributed and decentralized +nature of a Fabric blockchain network later in this section. ![Peer2](./peers.diagram.2.png) -*A peer hosts instances of ledgers and instances of chaincodes. In this example, -P1 hosts an instance of ledger L1 and an instance of chaincode S1. There -can be many ledgers and chaincodes hosted on an individual peer.* +*A peer hosts instances of ledgers and instances of chaincodes (above). In +this example, P1 hosts an instance of ledger L1 and an instance of chaincode S1. +There can be many ledgers and chaincodes hosted on any individual peer.* -Because a peer is a *host* for ledgers and chaincodes, applications and -administrators must interact with a peer if they want to access these resources. -That's why peers are considered the most fundamental building blocks of a -Fabric network. When a peer is first created, it has neither ledgers nor -chaincodes. We'll see later how ledgers get created, and how chaincodes get -installed, on peers. +Because a peer is a *host* for ledgers, chaincodes and services, client +applications and administrators must interact with a peer to access these resources. +That's why peers are considered the fundamental building blocks of a Fabric network. +We'll see later how ledgers get created, and how chaincodes get installed, on peers. ### Multiple Ledgers -A peer is able to host more than one ledger, which is helpful because it allows -for a flexible system design. The simplest configuration is for a peer to manage a -single ledger, but it's absolutely appropriate for a peer to host two or more -ledgers when required. +A peer is capable of hosting more than one ledger, which is useful because it allows +for a flexible system design where a single peer can belong to multiple channels in +a Fabric network. In the simplest configuration, a peer hosts a single ledger, and +therefore belongs to a single channel. But it is not uncommon for a peer to host +multiple ledgers because it belongs to multiple channels. ![Peer3](./peers.diagram.3.png) -*A peer hosting multiple ledgers. Peers host one or more ledgers, and each -ledger has zero or more chaincodes that apply to them. In this example, we -can see that the peer P1 hosts ledgers L1 and L2. Ledger L1 is accessed using -chaincode S1. Ledger L2 on the other hand can be accessed using chaincodes S1 and S2.* +*A peer hosting multiple ledgers (above). Peers host one or more ledgers and +the chaincodes that access them. In this example, peer P1 hosts ledgers +L1 and L2. Ledger L1 is accessed using chaincode S1; Ledger L2 can be accessed +using either chaincode S1 or S2.* -Although it is perfectly possible for a peer to host a ledger instance without -hosting any chaincodes which access that ledger, it's rare that peers are configured -this way. The vast majority of peers will have at least one chaincode installed -on it which can query or update the peer's ledger instances. It's worth -mentioning in passing that, whether or not users have installed chaincodes for use by -external applications, peers also have special **system chaincodes** that are -always present. These are not discussed in detail in this topic. +Chaincodes installed on a peer can query or update (write to) a ledger. Note +that peers also host special *system chaincodes* which relate to the overall +configuration of the Fabric network. ### Multiple Chaincodes -There isn't a fixed relationship between the number of ledgers a peer has and -the number of chaincodes that can access that ledger. A peer might have -many chaincodes and many ledgers available to it. +A chaincode is instantiated on a single channel. Each channel (and ledger) can +have multiple chaincodes that interact with it. ![Peer4](./peers.diagram.4.png) -*An example of a peer hosting multiple chaincodes. Each ledger can have -many chaincodes which access it. In this example, we can see that peer P1 -hosts ledgers L1 and L2, where L1 is accessed by chaincodes S1 and S2, and -L2 is accessed by S1 and S3. We can see that S1 can access both L1 and L2.* +*An example of a peer hosting multiple chaincodes (above). Each ledger can have +many chaincodes which access it. In this example, peer P1 hosts ledgers L1 and L2, +where L1 is accessed by chaincodes S1 and S2, and L2 is accessed by S1 and S3. S1 +can access both L1 and L2.* We'll see a little later why the concept of **channels** in Fabric is important -when hosting multiple ledgers or multiple chaincodes on a peer. +to hosting multiple ledgers and multiple chaincodes on a peer. + +## Fabric Gateway service + +Starting in Hyperledger Fabric v2.4, the [Fabric Gateway](../gateway.html) service +is installed and enabled on each peer by default. The gateway service, as opposed +to the client application (in Fabric v2.3 and earlier), manages transaction proposals +and endorsements on the peer. The Gateway SDKs (v1.0.0 for Go, Node and Java) incorporate +this peer-centric model of transaction processing, enabling simplified application +development. Client applications developed with Fabric v2.3 or earlier SDKs will +continue to run in Fabric v2.4. ## Applications and Peers -We're now going to show how applications interact with peers to access the -ledger. Ledger-query interactions involve a simple three-step dialogue between -an application and a peer; ledger-update interactions are a little more -involved, and require two extra steps. We've simplified these steps a little to -help you get started with Fabric, but don't worry --- what's most important to -understand is the difference in application-peer interactions for ledger-query -compared to ledger-update transaction styles. - -Applications always connect to peers when they need to access ledgers and -chaincodes. The Fabric Software Development Kit (SDK) makes this -easy for programmers --- its APIs enable applications to connect to peers, invoke -chaincodes to generate transactions, submit transactions to the network that -will get ordered, validated and committed to the distributed ledger, and receive -events when this process is complete. - -Through a peer connection, applications can execute chaincodes to query or -update a ledger. The result of a ledger query transaction is returned -immediately, whereas ledger updates involve a more complex interaction between -applications, peers and orderers. Let's investigate this in a little more detail. - -![Peer6](./peers.diagram.6.png) - -*Peers, in conjunction with orderers, ensure that the ledger is kept up-to-date -on every peer. In this example, application A connects to P1 and invokes -chaincode S1 to query or update the ledger L1. P1 invokes S1 to generate a -proposal response that contains a query result or a proposed ledger update. -Application A receives the proposal response and, for queries, -the process is now complete. For updates, A builds a transaction -from all of the responses, which it sends to O1 for ordering. O1 collects -transactions from across the network into blocks, and distributes these to all -peers, including P1. P1 validates the transaction before committing to L1. Once L1 -is updated, P1 generates an event, received by A, to signify completion.* - -A peer can return the results of a query to an application immediately since -all of the information required to satisfy the query is in the peer's local copy of -the ledger. Peers never consult with other peers in order to respond to a query from -an application. Applications can, however, connect to one or more peers to issue -a query; for example, to corroborate a result between multiple peers, or -retrieve a more up-to-date result from a different peer if there's a suspicion -that information might be out of date. In the diagram, you can see that ledger -query is a simple three-step process. - -An update transaction starts in the same way as a query transaction, but has two -extra steps. Although ledger-updating applications also connect to peers to -invoke a chaincode, unlike with ledger-querying applications, an individual peer -cannot perform a ledger update at this time, because other peers must first -agree to the change --- a process called **consensus**. Therefore, peers return -to the application a **proposed** update --- one that this peer would apply -subject to other peers' prior agreement. The first extra step --- step four --- -requires that applications send an appropriate set of matching proposed updates -to the entire network of peers as a transaction for commitment to their -respective ledgers. This is achieved by the application by using an **orderer** to -package transactions into blocks, and distributing them to the entire network of -peers, where they can be verified before being applied to each peer's local copy -of the ledger. As this whole ordering processing takes some time to complete -(seconds), the application is notified asynchronously, as shown in step five. - -Later in this section, you'll learn more about the detailed nature of this -ordering process --- and for a really detailed look at this process see the -[Transaction Flow](../txflow.html) topic. +We're now going to show how client applications interact with peers, and more +specifically, with the [Fabric Gateway](../gateway.html) service running on peers, +to access the ledger. Ledger-queries involve a simple dialogue between an +application and a peer, while ledger-updates (writes) involve additional steps. + +A client application connects to the [Fabric Gateway](../gateway.html) service +on a peer in order to access ledgers and chaincodes. Starting in Fabric v2.4, the Gateway SDKs +(v1.x) make this easy for programmers. The APIs enable applications, via +the gateway, to submit transaction proposals (which invokes chaincode), request +endorsement, receive events, and forward endorsed transactions to the ordering +service. + +Through a peer connection on the gateway, applications can run chaincodes to query or +update the ledger. The result of a ledger query transaction is returned with +simple processing, whereas a ledger update (write) involves a more complex workflow +between applications, peers and orderers. Let's investigate this ledger update process in +detail. + +Peers, in conjunction with orderers, ensure that the ledger is kept consistent +and current on every peer in a channel. The following sequence, in three +phases, describes how interactions between a client application, the gateway +service running on a peer, orderer nodes and additional peers update the ledger. + +**Attention:** The three transaction phases which follow explain the internal methods of how Fabric manages transactions. The Fabric Gateway SDKs implement these phases seamlessly; developers only need to use a [Gateway SDK (1.x)](../gateway.html). + +### Phase 1 - Transaction Proposal and Endorsement + +Phase 1 of a ledger update (write) consists of transaction proposal submission, execution and endorsement: + +a) **Transaction proposal** --- The client application (A1) submits a signed transaction proposal by connecting to the gateway service on P1. A1 must either delegate the selection of endorsing organizations to the gateway service or explicitly identify the organizations required for endorsement. + +b) **Transaction execution** --- The gateway service selects P1, or another peer in its organization, to execute the transaction. The selected peer executes the chaincode (S1) specified in the proposal, generates a proposal response (containing the read-write set). The selected peer signs the proposal response and returns it to the gateway. + +c) **Transaction endorsement** --- The gateway repeats transaction execution (b) for each organization required by the chaincode (smart contract) endorsement policies. The gateway service collects the signed proposal responses and creates a transaction envelope --- which it returns to the client (SDK) for signing. + +### Phase 2 - Transaction Submission and Ordering + +Phase 2 of a ledger update consists of transaction submission and ordering into blocks: + +a) **Transaction submission** --- The client (SDK) sends the signed transaction envelope to the gateway service. The gateway forwards the envelope to an ordering node and returns a success message to the client. + +b) **Transaction ordering** --- The ordering node (O1) verifies the signature, and the ordering service orders the transaction, and packages it with other ordered transactions into blocks. The ordering service then distributes the block to all peers in the channel for validation and commitment to the ledger. + +### Phase 3 - Transaction Validation and Commitment + +Phase 3 of a ledger update consists of transaction validation, ledger commitment and a commit event: + +a) **Transaction validation** --- Each peer checks that the client signature on the transaction envelope matches the signature on the original transaction proposal. Each peer also checks that +all read-write sets and status responses are equivalent (i.e. the endorsements from all peers match) and that the endorsements satisfy the endorsement policies. Each peer then marks each transaction as valid or invalid for commitment to the ledger. + +b) **Transaction commitment** --- Each peer commits the ordered block of transactions to the channel ledger (L1). The commit is an immutable ledger update (write) to the channel ledger. The world state (essentially, the sum of all valid transactions) of the channel is updated with results of valid transactions only. + +c) **Commit event** --- Each peer that commits to the ledger sends the client a commit status event with proof of the ledger update. + +Note: Fabric v2.3 SDKs, which embed transaction proposal and endorsement functionality in the client application, remain supported in Fabric v2.4. Refer to the [v2.3 Applications and Peers](https://hyperledger-fabric.readthedocs.io/en/release-2.3/peers/peers.html#applications-and-peers) topic for details. ## Peers and Channels -Although this section is about peers rather than channels, it's worth spending a -little time understanding how peers interact with each other, and with applications, -via *channels* --- a mechanism by which a set of components within a blockchain -network can communicate and transact *privately*. - -These components are typically peer nodes, orderer nodes and applications and, -by joining a channel, they agree to collaborate to collectively share and -manage identical copies of the ledger associated with that channel. Conceptually, you can -think of channels as being similar to groups of friends (though the members of a -channel certainly don't need to be friends!). A person might have several groups -of friends, with each group having activities they do together. These groups -might be totally separate (a group of work friends as compared to a group of -hobby friends), or there can be some crossover between them. Nevertheless, each group -is its own entity, with "rules" of a kind. +It is worth spending some time understanding how peers interact with each other, and +with applications, on a *channel* --- a mechanism by which components within a Fabric +blockchain network communicate and transact *privately*. + +Channel components include peer nodes, orderer nodes and applications, and +by joining a channel, they agree to collaborate to collectively manage and +share identical copies of the ledger. Conceptually, you can compare channels +to groups of friends; a person might have several groups of friends with each +group participating in different activities. These groups might be entirely +separate (a group of work friends as compared to a group of hobby friends), or there +can be some crossover membership between them. Nevertheless, each friend group +is its own entity, with specific rules (or expectations) that establish and +maintain membership. + +Channel membership works the same way as in other groups; any one peer may belong +to several channels and maintain a ledger and chaincodes specific to each channel. +Or a peer may belong to only a single channel, and therefore have only one set of +rules to follow. ![Peer5](./peers.diagram.5.png) -*Channels allow a specific set of peers and applications to communicate with -each other within a blockchain network. In this example, application A can -communicate directly with peers P1 and P2 using channel C. You can think of the -channel as a pathway for communications between particular applications and -peers. (For simplicity, orderers are not shown in this diagram, but must be -present in a functioning network.)* +*Channels allow a specific set of applications and peers (and organizations) +to communicate with each other on a Fabric blockchain network. +In the example above, application A communicates with peers P1 and P2, through +the gateway service, on channel C. The channel is a pathway for communications +between specific applications and peers. We see that channels don't exist in the same way that peers do --- it's more -appropriate to think of a channel as a logical structure that is formed by a +accurate to think of a channel as a logical structure that is formed by a collection of physical peers. *It is vital to understand this point --- peers provide the control point for access to, and management of, channels*. @@ -197,55 +204,45 @@ Now that you understand peers and their relationship to ledgers, chaincodes and channels, you'll be able to see how multiple organizations come together to form a blockchain network. -Blockchain networks are administered by a collection of organizations rather +Fabric blockchain networks are administered by a collection of organizations rather than a single organization. Peers are central to how this kind of distributed -network is built because they are owned by --- and are the connection points to -the network for --- these organizations. +network is built because they are owned by --- and are the network connection points +for --- these organizations. ![Peer8](./peers.diagram.8.png) -*Peers in a blockchain network with multiple organizations. The blockchain -network is built up from the peers owned and contributed by the different -organizations. In this example, we see four organizations contributing eight -peers to form a network. The channel C connects five of these peers in the -network N --- P1, P3, P5, P7 and P8. The other peers owned by these -organizations have not been joined to this channel, but are typically joined to -at least one other channel. Applications that have been developed by a -particular organization will connect to their own organization's peers as well -as those of different organizations. Again, -for simplicity, an orderer node is not shown in this diagram.* - -It's really important that you can see what's happening in the formation of a +*The example above shows organizations and their peers in a Fabric blockchain network. +We see four organizations contributing a total of eight peers to form a network. +Channel C connects five of these peers in the network N --- P1, P3, +P5, P7 and P8. The other peers owned by these organizations have not joined +channel C, but typically join at least one other channel. Applications +developed by an organization connect to peers, via the Fabric Gateway service, +in the same organization as well as peers in other organizations on a channel. + +It is important to notice what happens during the formation of a Fabric blockchain network. *The network is both formed and managed by the multiple -organizations who contribute resources to it.* Peers are the resources that -we're discussing in this topic, but the resources an organization provides are -more than just peers. There's a principle at work here --- the network literally +organizations that contribute resources to it.* Peers are the resources that +we're discussing in this topic, but organizations provide other resources too, +such as chaincode and ordering service nodes. There's a principle at work here --- the network literally does not exist without organizations contributing their individual resources to -the collective network. Moreover, the network grows and shrinks with the -resources that are provided by these collaborating organizations. - -You can see that (other than the ordering service) there are no centralized -resources --- in the [example above](#Peer8), the network, **N**, would not exist -if the organizations did not contribute their peers. This reflects the fact that -the network does not exist in any meaningful sense unless and until -organizations contribute the resources that form it. Moreover, the network does -not depend on any individual organization --- it will continue to exist as long -as one organization remains, no matter which other organizations may come and -go. This is at the heart of what it means for a network to be decentralized. - -Applications in different organizations, as in the [example above](#Peer8), may -or may not be the same. That's because it's entirely up to an organization as to how -its applications process their peers' copies of the ledger. This means that both -application and presentation logic may vary from organization to organization -even though their respective peers host exactly the same ledger data. - -Applications connect either to peers in their organization, or peers in another -organization, depending on the nature of the ledger interaction that's required. -For ledger-query interactions, applications typically connect to their own -organization's peers. For ledger-update interactions, we'll see later why -applications need to connect to peers representing *every* organization that is -required to endorse the ledger update. +the collective network. Moreover, the network grows as resources are provided +by collaborating organizations, increasing the resiliency and security of the +network. + +You can see that (other than the ordering service, to some degree) there are no +centralized resources --- in the [previous diagram](#Peer8), the network **N** +would not exist if the organizations did not contribute their peers and other +resources. Moreover, the network does not depend on any individual organization --- it +continues to exist despite departures as long as its membership meets the +self-defined requirements of the network. This is at the heart of what it means +for a network to be decentralized; all of its member organizations share and +contribute equally. + +Applications used by organizations, as in the [previous diagram](#Peer8), may or +may not be the same, because each organization can decide how it wants to use +data on the ledger. Both application and presentation logic can therefore vary +across organizations, even though all peers host an equivalent copy of the ledger. ## Peers and Identity @@ -254,181 +251,121 @@ form a blockchain network, it's worth spending a few moments understanding how peers get assigned to organizations by their administrators. Peers have an identity assigned to them via a digital certificate from a -particular certificate authority. You can read lots more about how X.509 -digital certificates work elsewhere in this guide but, for now, think of a -digital certificate as being like an ID card that provides lots of verifiable +particular certificate authority. You can read a lot more about how X.509 +digital certificates work elsewhere in this guide, but for now, think of a +digital certificate as like an ID card that provides verifiable information about a peer. *Each and every peer in the network is assigned a digital certificate by an administrator from its owning organization*. ![Peer9](./peers.diagram.9.png) *When a peer connects to a channel, its digital certificate identifies its -owning organization via a channel MSP. In this example, P1 and P2 have -identities issued by CA1. Channel C determines from a policy in its channel -configuration that identities from CA1 should be associated with Org1 using -ORG1.MSP. Similarly, P3 and P4 are identified by ORG2.MSP as being part of -Org2.* +owning organization via a channel MSP. In the example above, P1 and P2 +have identities issued by a certificate authority (CA1). Channel C determines, +from a policy in its channel configuration, that identities from CA1 should be +associated with Org1 using ORG1.MSP. Similarly, P3 and P4 are identified by +ORG2.MSP as part of Org2.* -Whenever a peer connects using a channel to a blockchain network, *a policy in +Whenever a peer connects to a Fabric network channel, *a policy in the channel configuration uses the peer's identity to determine its rights.* The mapping of identity to organization is provided by a component -called a *Membership Service Provider* (MSP) --- it determines how a peer gets -assigned to a specific role in a particular organization and accordingly gains -appropriate access to blockchain resources. Moreover, a peer can be owned only -by a single organization, and is therefore associated with a single MSP. We'll -learn more about peer access control later in this section, and there's an entire -section on MSPs and access control policies elsewhere in this guide. But for now, -think of an MSP as providing linkage between an individual identity and a -particular organizational role in a blockchain network. - -To digress for a moment, peers as well as *everything that interacts with a -blockchain network acquire their organizational identity from their digital -certificate and an MSP*. Peers, applications, end users, administrators and -orderers must have an identity and an associated MSP if they want to interact -with a blockchain network. *We give a name to every entity that interacts with -a blockchain network using an identity --- a principal.* You can learn lots -more about principals and organizations elsewhere in this guide, but for now -you know more than enough to continue your understanding of peers! - -Finally, note that it's not really important where the peer is physically -located --- it could reside in the cloud, or in a data centre owned by one -of the organizations, or on a local machine --- it's the digital certificate -associated with it that identifies it as being owned by a particular organization. -In our example above, P3 could be hosted in Org1's data center, but as long as the -digital certificate associated with it is issued by CA2, then it's owned by -Org2. +called a *Membership Service Provider* (MSP) --- which determines how a peer gets +assigned to a specific role in a particular organization and accordingly, gains +authorized access to resources. Moreover, a peer can be owned only +by a single organization, and is therefore associated with a single MSP. Think of +an MSP as linking an individual identity with a particular organizational role +in a Fabric network. + +Peers, and *anything that interacts with a Fabric network, acquire their +organizational identity from their digital certificate and an MSP*. Peers, +applications, end users, administrators and orderers must each have an +identity, and an associated MSP, to interact with the network. Any entity +that interacts with a blockchain network using an identity is known as a +*principal.* Note that identity is distinct from where the peer is physically +located --- it could reside in the cloud, or in a data center owned by one +of the organizations, or on a local machine --- the digital certificate +associated with the peer is what identifies it as owned by a particular +organization. In the previous example diagram, P3 could be hosted in Org1's data center, but +as long as the digital certificate associated with it is issued by CA2, it +is owned by Org2. ## Peers and Orderers -We've seen that peers form the basis for a blockchain network, hosting ledgers -and smart contracts which can be queried and updated by peer-connected applications. -However, the mechanism by which applications and peers interact with each other -to ensure that every peer's ledger is kept consistent with each other is mediated -by special nodes called *orderers*, and it's to these nodes we now turn our +We've seen that peers form the basis for a Fabric blockchain network, hosting +ledgers and smart contracts which can be queried and updated by peer-connected +applications. The mechanism by which applications and peers interact with each +other to keep the ledger current and consistent across a channel is mediated by +special nodes called *orderers*. It's to these orderer nodes that we now turn our attention. -An update transaction is quite different from a query transaction because a single -peer cannot, on its own, update the ledger --- updating requires the consent of other -peers in the network. A peer requires other peers in the network to approve a -ledger update before it can be applied to a peer's local ledger. This process is -called *consensus*, which takes much longer to complete than a simple query. But when -all the peers required to approve the transaction do so, and the transaction is -committed to the ledger, peers will notify their connected applications that the -ledger has been updated. You're about to be shown a lot more detail about how -peers and orderers manage the consensus process in this section. - -Specifically, applications that want to update the ledger are involved in a -3-phase process, which ensures that all the peers in a blockchain network keep -their ledgers consistent with each other. - -* In the first phase, applications work with a subset of *endorsing peers*, each of - which provide an endorsement of the proposed ledger update to the application, - but do not apply the proposed update to their copy of the ledger. -* In the second phase, these separate endorsements are collected together - as transactions and packaged into blocks. -* In the third and final phase, these blocks are distributed back to every peer where - each transaction is validated before being committed to that peer's copy of the ledger. - -As you will see, orderer nodes are central to this process, so let's -investigate in a little more detail how applications and peers use orderers to -generate ledger updates that can be consistently applied to a distributed, -replicated ledger. - -### Phase 1: Proposal - -Phase 1 of the transaction workflow involves an interaction between an -application and a set of peers --- it does not involve orderers. Phase 1 is only -concerned with an application asking different organizations' endorsing peers to -agree to the results of the proposed chaincode invocation. - -To start phase 1, applications generate a transaction proposal which they send -to each of the required set of peers for endorsement. Each of these *endorsing peers* then -independently executes a chaincode using the transaction proposal to -generate a transaction proposal response. It does not apply this update to the -ledger, but rather simply signs it and returns it to the application. Once the -application has received a sufficient number of signed proposal responses, -the first phase of the transaction flow is complete. Let's examine this phase in -a little more detail. - -![Peer10](./peers.diagram.10.png) - -*Transaction proposals are independently executed by peers who return endorsed -proposal responses. In this example, application A1 generates transaction T1 -proposal P which it sends to both peer P1 and peer P2 on channel C. P1 executes -S1 using transaction T1 proposal P generating transaction T1 response R1 which -it endorses with E1. Independently, P2 executes S1 using transaction T1 -proposal P generating transaction T1 response R2 which it endorses with E2. -Application A1 receives two endorsed responses for transaction T1, namely E1 -and E2.* - -Initially, a set of peers are chosen by the application to generate a set of -proposed ledger updates. Which peers are chosen by the application? Well, that -depends on the *endorsement policy* (defined for a chaincode), which defines -the set of organizations that need to endorse a proposed ledger change before it -can be accepted by the network. This is literally what it means to achieve -consensus --- every organization who matters must have endorsed the proposed -ledger change *before* it will be accepted onto any peer's ledger. - -A peer endorses a proposal response by adding its digital signature, and signing -the entire payload using its private key. This endorsement can be subsequently -used to prove that this organization's peer generated a particular response. In -our example, if peer P1 is owned by organization Org1, endorsement E1 -corresponds to a digital proof that "Transaction T1 response R1 on ledger L1 has -been provided by Org1's peer P1!". - -Phase 1 ends when the application receives signed proposal responses from -sufficient peers. We note that different peers can return different and -therefore inconsistent transaction responses to the application *for the same -transaction proposal*. It might simply be that the result was generated at -different times on different peers with ledgers at different states, in which -case an application can simply request a more up-to-date proposal response. Less -likely, but much more seriously, results might be different because the chaincode -is *non-deterministic*. Non-determinism is the enemy of chaincodes -and ledgers and if it occurs it indicates a serious problem with the proposed -transaction, as inconsistent results cannot, obviously, be applied to ledgers. -An individual peer cannot know that their transaction result is -non-deterministic --- transaction responses must be gathered together for -comparison before non-determinism can be detected. (Strictly speaking, even this -is not enough, but we defer this discussion to the transaction section, where -non-determinism is discussed in detail.) - -At the end of phase 1, the application is free to discard inconsistent -transaction responses if it wishes to do so, effectively terminating the -transaction workflow early. We'll see later that if an application tries to use -an inconsistent set of transaction responses to update the ledger, it will be -rejected. - -### Phase 2: Ordering and packaging transactions into blocks - -The second phase of the transaction workflow is the packaging phase. The orderer -is pivotal to this process --- it receives transactions containing endorsed -transaction proposal responses from many applications, and orders the -transactions into blocks. For more details about the -ordering and packaging phase, check out our -[conceptual information about the ordering phase](../orderer/ordering_service.html#phase-two-ordering-and-packaging-transactions-into-blocks). - -### Phase 3: Validation and commit - -At the end of phase 2, we see that orderers have been responsible for the simple -but vital processes of collecting proposed transaction updates, ordering them, -and packaging them into blocks, ready for distribution to the peers. - -The final phase of the transaction workflow involves the distribution and -subsequent validation of blocks from the orderer to the peers, where they can be -committed to the ledger. Specifically, at each peer, every transaction within a -block is validated to ensure that it has been consistently endorsed by all -relevant organizations before it is committed to the ledger. Failed transactions -are retained for audit, but are not committed to the ledger. +A ledger update transaction is different from a query transaction because a single +peer cannot, on its own, update the ledger --- that requires the consent of other +peers in the network, a process known as *consensus*. When the peers required to +approve the transaction do so, and the transaction is committed to the ledger, +the gateway service notifies the applications that the ledger has been updated. +Peers and orderers work together to manage consensus. + +### Orderers and the Three Phases + +Previously, in the [Applications and Peers](#peers-and-application) topic, we +saw how a client application that requests a ledger update initiates a three-phase +process which, with help from the Fabric Gateway service, ensures that +all peers in a Fabric network maintain current and consistent copies of the ledger. +As described, orderers are also involved; the following sections take a closer +look at the three-phase process from the perspective of ordering nodes and peers +(i.e. the ordering service and the gateway service). + +### Phase 1: Transaction Proposal + +Phase 1 of the transaction workflow involves an interaction between a client +application, peers and orderers. In Phase 1, the client application initiates +a request to the Fabric Gateway service to evaluate a transaction proposal. + +The target peer, selected by the client application, executes the transaction +by invoking chaincode --- this step can be described as simulating the transaction, +because it runs the transaction without any effect on the ledger. The peer then +returns its transaction result to the client. + +The gateway service also forwards the transaction proposal to the required +*endorsing peers* (based on the endorsement policies), which also execute the +transaction and return their results to the peer. The gateway service collects +all responses, and if they collectively satisfy the endorsement policies, forwards +the transaction to the ordering service. + +If the proposed transaction would write to a private data collection, (as *transient* data) +the client application must explicitly specify the organizations required for endorsement. + +Note that a peer endorses a proposal response by adding its digital signature, and +signing the entire payload using its private key. This endorsement can be subsequently +used to prove that this organization's peer generated a particular response. + +### Phase 2: Transaction Ordering + +The second phase of the transaction workflow is the ordering and packaging phase. +The ordering service (running on orderer nodes) receives transactions containing signed and +endorsed proposal responses, from one or more applications via the gateway service, and +orders and packages the transactions into blocks. These are the blocks (which are also ordered) --- consisting +of endorsed and ordered transactions --- that make up a Fabric blockchain ledger. + +For details about the ordering and packaging phase, see +the [conceptual information about the ordering phase](../orderer/ordering_service.html#phase-two-ordering-and-packaging-transactions-into-blocks). + +### Phase 3: Validation and Commitment + +The third and final phase of the transaction workflow is the distribution of ordered +transactions from orderers to peers. Each peer then validates each transaction, in +the correct order, and ensures that each transaction has been consistently endorsed by all +required organizations. Only then does the peer commit the block to its copy of the channel ledger. ![Peer12](./peers.diagram.12.png) -*The second role of an orderer node is to distribute blocks to peers. In this -example, orderer O1 distributes block B2 to peer P1 and peer P2. Peer P1 +*An orderer node distributes ordered blocks to peers for validation and +commitment. In this example above, orderer O1 distributes block B2 to peers P1 and P2. Peer P1 processes block B2, resulting in a new block being added to ledger L1 on P1. In parallel, peer P2 processes block B2, resulting in a new block being added -to ledger L1 on P2. Once this process is complete, the ledger L1 has been -consistently updated on peers P1 and P2, and each may inform connected -applications that the transaction has been processed.* +to ledger L1 on P2. Once complete, the ledger L1 has been consistently updated +on peers P1 and P2, and the gateway service informs the +relevant applications that the transaction has been committed.* Phase 3 begins with the orderer distributing blocks to all peers connected to it. Peers are connected to orderers on channels such that when a new block is @@ -440,19 +377,13 @@ to be connected to an orderer --- peers can cascade blocks to other peers using the **gossip** protocol, who also can process them independently. But let's leave that discussion to another time! -Upon receipt of a block, a peer will process each transaction in the sequence in -which it appears in the block. For every transaction, each peer will verify that +Upon receipt of a block, a peer will process each transaction in the sequence +specified in the block. For each transaction, the peer will verify that the transaction has been endorsed by the required organizations according to the -*endorsement policy* of the chaincode which generated the transaction. For +*endorsement policies* for the chaincode which generated the transaction. For example, some transactions may only need to be endorsed by a single -organization, whereas others may require multiple endorsements before they are -considered valid. This process of validation verifies that all relevant -organizations have generated the same outcome or result. Also note that this -validation is different than the endorsement check in phase 1, where it is the -application that receives the response from endorsing peers and makes the -decision to send the proposal transactions. In case the application violates -the endorsement policy by sending wrong transactions, the peer is still able to -reject the transaction in the validation process of phase 3. +organization, whereas others may require multiple endorsements to be valid. This process of +validation verifies that all relevant organizations have generated the same outcome or result. If a transaction has been endorsed correctly, the peer will attempt to apply it to the ledger. To do this, a peer must perform a ledger consistency check to @@ -470,46 +401,46 @@ retained for audit purposes, as are successful transactions. This means that peer blocks are almost exactly the same as the blocks received from the orderer, except for a valid or invalid indicator on each transaction in the block. -We also note that phase 3 does not require the running of chaincodes --- this is +Note that phase 3 does not require running chaincode --- this is done only during phase 1, and that's important. It means that chaincodes only have to be available on endorsing nodes, rather than throughout the blockchain -network. This is often helpful as it keeps the logic of the chaincode -confidential to endorsing organizations. This is in contrast to the output of -the chaincodes (the transaction proposal responses) which are shared with every +network. This keeps the logic of the chaincode confidential to endorsing organizations +only. This is in contrast to the output of +the chaincodes (transaction proposal responses), which are shared with every peer in the channel, whether or not they endorsed the transaction. This specialization of endorsing peers is designed to help scalability and confidentiality. -Finally, every time a block is committed to a peer's ledger, that peer -generates an appropriate *event*. *Block events* include the full block content, +Finally, each time a block is committed to a peer's ledger, that peer +generates an *event*. *Block events* include the full block content, while *block transaction events* include summary information only, such as whether each transaction in the block has been validated or invalidated. *Chaincode* events that the chaincode execution has produced can also be -published at this time. Applications can register for these event types so -that they can be notified when they occur. These notifications conclude the -third and final phase of the transaction workflow. +published at this time. Applications can register for these event types +for notification. The commit event notification concludes the third and final +phase of the transaction workflow. -In summary, phase 3 sees the blocks which are generated by the orderer -consistently applied to the ledger. The strict ordering of transactions into -blocks allows each peer to validate that transaction updates are consistently -applied across the blockchain network. +In summary, phase 3 sees the transaction blocks which are generated by the orderer +validated and consistently applied to the ledger. The strict ordering of transactions +into blocks allows each peer to validate that transaction updates are consistently +applied across the channel. ### Orderers and Consensus -This entire transaction workflow process is called *consensus* because all peers -have reached agreement on the order and content of transactions, in a process -that is mediated by orderers. Consensus is a multi-step process and applications -are only notified of ledger updates when the process is complete --- which may -happen at slightly different times on different peers. - -We will discuss orderers in a lot more detail in a future orderer topic, but for -now, think of orderers as nodes which collect and distribute proposed ledger -updates from applications for peers to validate and include on the ledger. - -That's it! We've now finished our tour of peers and the other components that -they relate to in Fabric. We've seen that peers are in many ways the -most fundamental element --- they form the network, host chaincodes and the -ledger, handle transaction proposals and responses, and keep the ledger -up-to-date by consistently applying transaction updates to it. +This entire transaction workflow process is called *consensus*, because peers have +to collectively reach agreement on the order and content of transactions, with +mediation from orderers. Consensus is a multi-step process and ledger updates +only occur when the process completes successfully --- which can happen at slightly +different times on different peers. Think of the orderers as nodes which sequence +and distribute proposed ledger updates for peers to endorse and add to the ledger. + +## Peers Summary + +That's it! We've now finished our detailed tour of peers and their interactions +with other Fabric components, including client applications, chaincodes and orderers. We've seen +that peers are in many ways the most fundamental element of Hyperledger Fabric +architecture --- they form the network, host chaincodes and the ledger, handle +transaction proposals and responses, and keep the ledger updated and consistent +with validated and endorsed transactions.