From 4ee7ce1361f2c530f75efbc26ba06d4c00511ea2 Mon Sep 17 00:00:00 2001 From: Nick Gaski Date: Tue, 20 Jun 2017 18:06:47 -0400 Subject: [PATCH] WIP [FAB-4821] - Add configtxlator to RTD WIP effort to eliminate requirement to build fabric from source incorporates the configtxlator README into the published docs. Conversion to .rst and formatting + grammar add orderer to release distribution? update samples.rst according with makefile change Fix merge Change-Id: I95861591c6541579eb72d4c342e38d63f1c602cb Signed-off-by: Nick Gaski Signed-off-by: Christopher Ferris --- Makefile | 2 +- docs/source/configtxlator.rst | 292 ++++++++++++++++++++++++++++++++++ docs/source/index.rst | 1 + 3 files changed, 294 insertions(+), 1 deletion(-) create mode 100644 docs/source/configtxlator.rst diff --git a/Makefile b/Makefile index c6d5228839b..201bf77ee72 100755 --- a/Makefile +++ b/Makefile @@ -82,7 +82,7 @@ PROJECT_FILES = $(shell git ls-files | grep -v ^test | grep -v ^unit-test | \ RELEASE_TEMPLATES = $(shell git ls-files | grep "release/templates") IMAGES = peer orderer ccenv javaenv buildenv testenv zookeeper kafka couchdb tools RELEASE_PLATFORMS = windows-amd64 darwin-amd64 linux-amd64 linux-ppc64le linux-s390x -RELEASE_PKGS = configtxgen cryptogen configtxlator peer +RELEASE_PKGS = configtxgen cryptogen configtxlator peer orderer pkgmap.cryptogen := $(PKGNAME)/common/tools/cryptogen pkgmap.configtxgen := $(PKGNAME)/common/configtx/tool/configtxgen diff --git a/docs/source/configtxlator.rst b/docs/source/configtxlator.rst new file mode 100644 index 00000000000..a23c9b0d544 --- /dev/null +++ b/docs/source/configtxlator.rst @@ -0,0 +1,292 @@ +Reconfiguring with configtxlator +================================ + +Overview +-------- + +The ``configtxlator`` tool was created to support reconfiguration independent +of SDKs. Channel configuration is stored as a transaction in configuration +blocks of a channel and may be manipulated directly, such as in the bdd behave +tests. However, at the time of this writing, no SDK natively supports +manipulating the configuration directly, so the ``configtxlator`` tool is +designed to provide an API which consumers of any SDK may interact with to +assist with configuration updates. + +The tool name is a portmanteau of *configtx* and *translator* and is intended to +convey that the tool simply converts between different equivalent data +representations. It does not generate configuration. It does not submit or +retrieve configuration. It does not modify configuration itself, it simply +provides some bijective operations between different views of the configtx +format. + +The standard usage is expected to be: + + 1. SDK retrieves latest config + 2. ``configtxlator`` produces human readable version of config + 3. User or application edits the config + 4. ``configtxlator`` is used to compute config update representation of + changes to the config + 5. SDK submits signs and submits config + +The ``configtxlator`` tool exposes a truly stateless REST API for interacting +with configuration elements. These REST components support converting the +native configuration format to/from a human readable JSON representation, as +well as computing configuration updates based on the difference between two +configurations. + +Because the ``configtxlator`` service deliberately does not contain any crypto +material, or otherwise secret information, it does not include any authorization +or access control. The anticipated typical deployment would be to operate as +a sandboxed container, locally with the application, so that there is a +dedicated ``configtxlator`` process for each consumer of it. + +Running the configtxlator +------------------------- + +The ``configtxlator`` tool can be downloaded with the other Hyperledger Fabric +platform-specific binaries. Please see :ref:`download-platform-specific-binaries` +for details. + +The tool may be configured to listen on a different port and you may also +specify the hostname using the ``--port`` and ``--hostname`` flags. To explore +the complete set of commands and flags, run ``configtxlator --help``. + +The binary will start an http server listening on the designated port and is now +ready to process request. + +To start the ``configtxlator`` server: + +.. code:: bash + + configtxlator start + 2017-06-21 18:16:58.248 HKT [configtxlator] startServer -> INFO 001 Serving HTTP requests on 0.0.0.0:7059 + +Proto translation +----------------- + +For extensibility, and because certain fields must be signed over, many proto +fields are stored as bytes. This makes the natural proto to JSON translation +using the ``jsonpb`` package ineffective for producing a human readable version +of the protobufs. Instead, the ``configtxlator`` exposes a REST component to do +a more sophisticated translation. + +To convert a proto to its human readable JSON equivalent, simply post the binary +proto to the rest target +``http://$SERVER:$PORT/protolator/decode/``, +where ```` is the fully qualified proto name of the message. + +For instance, to decode a configuration block saved as +``configuration_block.pb``, run the command: + +.. code:: bash + + curl -X POST --data-binary @configuration_block.pb http://127.0.0.1:7059/protolator/decode/common.Block + +To convert the human readable JSON version of the proto message, simply post the +JSON version to ``http://$SERVER:$PORT/protolator/encode/`` is again the fully qualified proto name of the message. + +For instance, to re-encode the block saved as ``configuration_block.json``, run +the command: + +.. code:: bash + + curl -X POST --data-binary @configuration_block.json http://127.0.0.1:7059/protolator/encode/common.Block + +Any of the configuration related protos, including ``common.Block``, +``common.Envelope``, ``common.ConfigEnvelope``, ``common.ConfigUpdateEnvelope``, +``common.Configuration``, and ``common.ConfigUpdate`` are valid targets for +these URLs. In the future, other proto decoding types may be added, such as +for endorser transactions. + +Config update computation +------------------------- + +Given two different configurations, it is possible to compute the config update +which transitions between them. Simply POST the two ``common.Config`` proto +encoded configurations as ``multipart/formdata``, with the original as field +``original`` and the updated as field ``updated``, to +``http://$SERVER:$PORT/configtxlator/compute/update-from-configs``. + +For example, given the original config as the file ``original_config.pb`` and +the updated config as the file ``updated_config.pb`` for the channel +``desiredchannel``: + +.. code:: bash + + curl -X POST -F channel=desiredchannel -F original=@original_config.pb -F updated=@updated_config.pb http://127.0.0.1:7059/configtxlator/compute/update-from-configs + +Bootstraping example +-------------------- + +First start the ``configtxlator``: + +.. code:: bash + + $ configtxlator start + 2017-05-31 12:57:22.499 EDT [configtxlator] main -> INFO 001 Serving HTTP requests on port: 7059 + +First, produce a genesis block for the ordering system channel: + +.. code:: bash + + $ configtxgen -outputBlock genesis_block.pb + 2017-05-31 14:15:16.634 EDT [common/configtx/tool] main -> INFO 001 Loading configuration + 2017-05-31 14:15:16.646 EDT [common/configtx/tool] doOutputBlock -> INFO 002 Generating genesis block + 2017-05-31 14:15:16.646 EDT [common/configtx/tool] doOutputBlock -> INFO 003 Writing genesis block + +Decode the genesis block into a human editable form: + +.. code:: bash + + curl -X POST --data-binary @genesis_block.pb http://127.0.0.1:7059/protolator/decode/common.Block > genesis_block.json + +Edit the ``genesis_block.json`` file in your favorite JSON editor, or manipulate +it programatically. Here we use the JSON CLI tool ``jq``. For simplicity, we +are editing the batch size for the channel, because it is a single numeric +field. However, any edits, including policy and MSP edits may be made here. + +First, let's establish an environment variable to hold the string that defines +the path to a property in the json: + +.. code:: bash + + export MAXBATCHSIZEPATH=".data.data[0].payload.data.config.channel_group.groups.Orderer.values.BatchSize.value.max_message_count" + +Next, let's display the value of that property: + +.. code:: bash + + jq "$MAXBATCHSIZEPATH" genesis_block.json + 10 + +Now, let's set the new batch size, and display the new value: + + jq "$MAXBATCHSIZEPATH = 20" genesis_block.json > updated_genesis_block.json + jq "$MAXBATCHSIZEPATH" updated_genesis_block.json + 20 + +The genesis block is now ready to be re-encoded into the native proto form to be +used for bootstrapping: + +.. code:: bash + + curl -X POST --data-binary @updated_genesis_block.json http://127.0.0.1:7059/protolator/encode/common.Block > updated_genesis_block.pb + +The ``updated_genesis_block.pb`` file may now be used as the genesis block for +bootstrapping an ordering system channel. + +Reconfiguration example +----------------------- + +In another terminal window, start the orderer using the default options, +including the provisional bootstrapper which will create a ``testchainid`` +ordering system channel. + +.. code:: bash + + ORDERER_GENERAL_LOGLEVEL=debug orderer + +Reconfiguring a channel can be performed in a very similar way to modifying a +genesis config. + +First, fetch the config_block proto: + +.. code:: bash + + $ peer channel fetch config config_block.pb -o 127.0.0.1:7050 -c testchainid + 2017-05-31 15:11:37.617 EDT [msp] getMspConfig -> INFO 001 intermediate certs folder not found at [/home/yellickj/go/src/github.com/hyperledger/fabric/sampleconfig/msp/intermediatecerts]. Skipping.: [stat /home/yellickj/go/src/github.com/hyperledger/fabric/sampleconfig/msp/intermediatecerts: no such file or directory] + 2017-05-31 15:11:37.617 EDT [msp] getMspConfig -> INFO 002 crls folder not found at [/home/yellickj/go/src/github.com/hyperledger/fabric/sampleconfig/msp/intermediatecerts]. Skipping.: [stat /home/yellickj/go/src/github.com/hyperledger/fabric/sampleconfig/msp/crls: no such file or directory] + Received block: 1 + Received block: 1 + 2017-05-31 15:11:37.635 EDT [main] main -> INFO 003 Exiting..... + +Next, send the config block to the ``configtxlator`` service for decoding: + +.. code:: bash + + curl -X POST --data-binary @config_block.pb http://127.0.0.1:7059/protolator/decode/common.Block > config_block.json + +Extract the config section from the block: + +.. code:: bash + + jq .data.data[0].payload.data.config config_block.json > config.json + +Edit the config, saving it as a new ``updated_config.json``. Here, we set the +batch size to 30. + +.. code:: bash + + jq ".channel_group.groups.Orderer.values.BatchSize.value.max_message_count = 30" config.json > updated_config.json + +Re-encode both the original config, and the updated config into proto: + +.. code:: bash + + curl -X POST --data-binary @config.json http://127.0.0.1:7059/protolator/encode/common.Config > config.pb + +.. code:: bash + + curl -X POST --data-binary @updated_config.json http://127.0.0.1:7059/protolator/encode/common.Config > updated_config.pb + +Now, with both configs properly encoded, send them to the `configtxlator` +service to compute the config update which transitions between the two. + +.. code:: bash + + curl -X POST -F original=@config.pb -F updated=@updated_config.pb http://127.0.0.1:7059/configtxlator/compute/update-from-configs -F channel=testchainid > config_update.pb + +At this point, the computed config update is now prepared. Traditionally, +an SDK would be used to sign and wrap this message. However, in the interest of +using only the peer cli, the `configtxlator` can also be used for this task. + +First, we decode the ConfigUpdate so that we may work with it as text: + +.. code:: bash + + $ curl -X POST --data-binary @config_update.pb http://127.0.0.1:7059/protolator/decode/common.ConfigUpdate > config_update.json + +Then, we wrap it in an envelope message: + +.. code:: bash + + echo '{"payload":{"header":{"channel_header":{"channel_id":"testchainid", "type":2}},"data":{"config_update":'$(cat config_update.json)'}}}' > config_update_as_envelope.json + +Next, convert it back into the proto form of a full fledged config +transaction: + +.. code:: bash + + curl -X POST --data-binary @config_update_as_envelope.json http://127.0.0.1:7059/protolator/encode/common.Envelope > config_update_as_envelope.pb + +Finally, submit the config update transaction to ordering to perform a config +update. + +.. code:: bash + + peer channel update -f config_update_as_envelope.pb -c testchainid -o 127.0.0.1:7050 + +Adding an organization +---------------------- + +First start the ``configtxlator``: + +.. code:: bash + + $ configtxlator start + 2017-05-31 12:57:22.499 EDT [configtxlator] main -> INFO 001 Serving HTTP requests on port: 7059 + +Start the orderer using the ``SampleDevModeSolo`` profile option. + +.. code:: bash + + ORDERER_GENERAL_LOGLEVEL=debug ORDERER_GENERAL_GENESISPROFILE=SampleDevModeSolo orderer + +The process to add an organization then follows exactly like the batch size +example. However, instead of setting the batch size, a new org is defined at +the application level. Adding an organization is slightly more involved because +we must first create a channel, then modify its membership set. + +.. Licensed under Creative Commons Attribution 4.0 International License + https://creativecommons.org/licenses/by/4.0/ diff --git a/docs/source/index.rst b/docs/source/index.rst index 45e497ec9fd..b8e8a8303ff 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -58,6 +58,7 @@ Before diving in, watch how Fabric is `Building a Blockchain for Business msp configtx configtxgen + configtxlator cc-packaging-and-signing endorsement-policies error-handling