From f26477c99db4e7db557382464637a74f2689520a Mon Sep 17 00:00:00 2001 From: Yuki Kondo Date: Wed, 29 Aug 2018 23:26:05 +0000 Subject: [PATCH] [FAB-11796]high-throughput:Remove unnecessary prunesafe fabric-samples/high-throughput is a sample Chaincode for delta-based transaction model. When executing `update`, a new delta for a particular variable is updated to the ledger. The sum of a variable is updated to ledger by deleting all of its delta rows while computing the sum. The current Chaincode has two types of pruning functions. However, there is no difference in terms of Fabric's transaction model. This CR adds the following changes. - Remove unnecessary `pruneSafe` function. - Change the name of function from `pruneFast` to `prune`. - Update or delete related scripts. - Improve a related documentation. FAB-11796 #done Change-Id: I5daa21554e53d77b7b5081f02a2846a85ec06f9a Signed-off-by: Yuki Kondo --- high-throughput/README.md | 19 ++-- high-throughput/chaincode/high-throughput.go | 98 ++----------------- high-throughput/scripts/install-chaincode.sh | 8 +- .../{prunefast-invoke.sh => prune-invoke.sh} | 2 +- high-throughput/scripts/prunesafe-invoke.sh | 8 -- 5 files changed, 23 insertions(+), 112 deletions(-) rename high-throughput/scripts/{prunefast-invoke.sh => prune-invoke.sh} (85%) delete mode 100755 high-throughput/scripts/prunesafe-invoke.sh diff --git a/high-throughput/README.md b/high-throughput/README.md index a899792fef..8ce49eb67b 100644 --- a/high-throughput/README.md +++ b/high-throughput/README.md @@ -104,19 +104,19 @@ and run some invocations are provided below. * In the `volumes` section of the `cli` container, edit the second line which refers to the chaincode folder to point to the chaincode folder within the `high-throughput` folder, e.g. - `./../chaincode/:/opt/gopath/src/github.com/hyperledger/fabric/examples/chaincode/go` --> - `./../high-throughput/chaincode/:/opt/gopath/src/github.com/hyperledger/fabric/examples/chaincode/go` + `./../chaincode/:/opt/gopath/src/github.com/hyperledger/fabric-samples/chaincode` --> + `./../high-throughput/chaincode/:/opt/gopath/src/github.com/hyperledger/fabric-samples/chaincode` * Again in the `volumes` section, edit the fourth line which refers to the scripts folder so it points to the scripts folder within the `high-throughput` folder, e.g. - `./scripts:/opt/gopath/src/github.com/hyperledger/fabric/peer/scripts/` --> + `./scripts:/opt/gopath/src/github.com/hyperledger/fabric/peer/scripts/` --> `./../high-throughput/scripts/:/opt/gopath/src/github.com/hyperledger/fabric/peer/scripts/` * Finally, comment out the `docker exec cli scripts/script.sh` command from the `byfn.sh` script by placing a `#` before it so that the standard BYFN end to end script doesn't run, e.g. `# docker exec cli scripts/script.sh $CHANNEL_NAME $CLI_DELAY $LANGUAGE $CLI_TIMEOUT $VERBOSE` -3. We can now bring our network up by typing in `./byfn.sh -m up -c mychannel` +3. We can now bring our network up by typing in `./byfn.sh up -c mychannel` 4. Open a new terminal window and enter the CLI container using `docker exec -it cli bash`, all operations on the network will happen within this container from now on. @@ -153,13 +153,11 @@ Example: `./delete-invoke.sh myvar` #### Prune Pruning takes all the deltas generated for a variable and combines them all into a single row, deleting all previous rows. This helps cleanup -the ledger when many updates have been performed. There are two types of pruning: `prunefast` and `prunesafe`. Prune fast performs the deletion -and aggregation simultaneously, so if an error happens along the way data integrity is not guaranteed. Prune safe performs the aggregation first, -backs up the results, then performs the deletion. This way, if an error occurs along the way, data integrity is maintained. +the ledger when many updates have been performed. -The format for pruning is: `./[prunesafe|prunefast]-invoke.sh name` where `name` is the name of the variable to prune. +The format for pruning is: `./prune-invoke.sh name` where `name` is the name of the variable to prune. -Example: `./prunefast-invoke.sh myvar` or `./prunesafe-invoke.sh myvar` +Example: `./prune-invoke.sh myvar` ### Test the Network Two scripts are provided to show the advantage of using this system when running many parallel transactions at once: `many-updates.sh` and @@ -175,5 +173,6 @@ errors in the peer and orderer logs. There is one other script, `get-traditional.sh`, which simply gets the value of a row in the traditional way, with no deltas. Examples: -`./many-updates.sh testvar 100 +` --> final value from `./get-invoke.sh` should be 100000 +`./many-updates.sh testvar 100 +` --> final value from `./get-invoke.sh testvar` should be 100000 + `./many-updates-traditional.sh testvar` --> final value from `./get-traditional.sh testvar` is undefined diff --git a/high-throughput/chaincode/high-throughput.go b/high-throughput/chaincode/high-throughput.go index 252c980548..00997a65dd 100644 --- a/high-throughput/chaincode/high-throughput.go +++ b/high-throughput/chaincode/high-throughput.go @@ -52,8 +52,7 @@ func (s *SmartContract) Init(APIstub shim.ChaincodeStubInterface) sc.Response { // Current supported invocations are: // - update, adds a delta to an aggregate variable in the ledger, all variables are assumed to start at 0 // - get, retrieves the aggregate value of a variable in the ledger -// - pruneFast, deletes all rows associated with the variable and replaces them with a single row containing the aggregate value -// - pruneSafe, same as pruneFast except it pre-computed the value and backs it up before performing any destructive operations +// - prune, deletes all rows associated with the variable and replaces them with a single row containing the aggregate value // - delete, removes all rows associated with the variable func (s *SmartContract) Invoke(APIstub shim.ChaincodeStubInterface) sc.Response { // Retrieve the requested Smart Contract function and arguments @@ -64,10 +63,8 @@ func (s *SmartContract) Invoke(APIstub shim.ChaincodeStubInterface) sc.Response return s.update(APIstub, args) } else if function == "get" { return s.get(APIstub, args) - } else if function == "prunefast" { - return s.pruneFast(APIstub, args) - } else if function == "prunesafe" { - return s.pruneSafe(APIstub, args) + } else if function == "prune" { + return s.prune(APIstub, args) } else if function == "delete" { return s.delete(APIstub, args) } else if function == "putstandard" { @@ -202,17 +199,15 @@ func (s *SmartContract) get(APIstub shim.ChaincodeStubInterface, args []string) /** * Prunes a variable by deleting all of its delta rows while computing the final value. Once all rows * have been processed and deleted, a single new row is added which defines a delta containing the final - * computed value of the variable. This function is NOT safe as any failures or errors during pruning - * will result in an undefined final value for the variable and loss of data. Use pruneSafe if data - * integrity is important. The args array contains the following argument: + * computed value of the variable. The args array contains the following argument: * - args[0] -> The name of the variable to prune * * @param APIstub The chaincode shim - * @param args The args array for the pruneFast invocation + * @param args The args array for the prune invocation * * @return A response structure indicating success or failure with a message */ -func (s *SmartContract) pruneFast(APIstub shim.ChaincodeStubInterface, args []string) sc.Response { +func (s *SmartContract) prune(APIstub shim.ChaincodeStubInterface, args []string) sc.Response { // Check we have a valid number of ars if len(args) != 1 { return shim.Error("Incorrect number of arguments, expecting 1") @@ -276,88 +271,13 @@ func (s *SmartContract) pruneFast(APIstub shim.ChaincodeStubInterface, args []st } } - // Update the ledger with the final value and return + // Update the ledger with the final value updateResp := s.update(APIstub, []string{name, strconv.FormatFloat(finalVal, 'f', -1, 64), "+"}) - if updateResp.Status == OK { - return shim.Success([]byte(fmt.Sprintf("Successfully pruned variable %s, final value is %f, %d rows pruned", args[0], finalVal, i))) - } - - return shim.Error(fmt.Sprintf("Failed to prune variable: all rows deleted but could not update value to %f, variable no longer exists in ledger", finalVal)) -} - -/** - * This function performs the same function as pruneFast except it provides data backups in case the - * prune fails. The final aggregate value is computed before any deletion occurs and is backed up - * to a new row. This back-up row is deleted only after the new aggregate delta has been successfully - * written to the ledger. The args array contains the following argument: - * args[0] -> The name of the variable to prune - * - * @param APIstub The chaincode shim - * @param args The arguments array for the pruneSafe invocation - * - * @result A response structure indicating success or failure with a message - */ -func (s *SmartContract) pruneSafe(APIstub shim.ChaincodeStubInterface, args []string) sc.Response { - // Verify there are a correct number of arguments - if len(args) != 1 { - return shim.Error("Incorrect number of arguments, expecting 1 (the name of the variable to prune)") - } - - // Get the var name - name := args[0] - - // Get the var's value and process it - getResp := s.get(APIstub, args) - if getResp.Status == ERROR { - return shim.Error(fmt.Sprintf("Could not retrieve the value of %s before pruning, pruning aborted: %s", name, getResp.Message)) - } - - valueStr := string(getResp.Payload) - val, convErr := strconv.ParseFloat(valueStr, 64) - if convErr != nil { - return shim.Error(fmt.Sprintf("Could not convert the value of %s to a number before pruning, pruning aborted: %s", name, convErr.Error())) - } - - // Store the var's value temporarily - backupPutErr := APIstub.PutState(fmt.Sprintf("%s_PRUNE_BACKUP", name), []byte(valueStr)) - if backupPutErr != nil { - return shim.Error(fmt.Sprintf("Could not backup the value of %s before pruning, pruning aborted: %s", name, backupPutErr.Error())) - } - - // Get all deltas for the variable - deltaResultsIterator, deltaErr := APIstub.GetStateByPartialCompositeKey("varName~op~value~txID", []string{name}) - if deltaErr != nil { - return shim.Error(fmt.Sprintf("Could not retrieve value for %s: %s", name, deltaErr.Error())) - } - defer deltaResultsIterator.Close() - - // Delete each row - var i int - for i = 0; deltaResultsIterator.HasNext(); i++ { - responseRange, nextErr := deltaResultsIterator.Next() - if nextErr != nil { - return shim.Error(fmt.Sprintf("Could not retrieve next row for pruning: %s", nextErr.Error())) - } - - deltaRowDelErr := APIstub.DelState(responseRange.Key) - if deltaRowDelErr != nil { - return shim.Error(fmt.Sprintf("Could not delete delta row: %s", deltaRowDelErr.Error())) - } - } - - // Insert new row for the final value - updateResp := s.update(APIstub, []string{name, valueStr, "+"}) if updateResp.Status == ERROR { - return shim.Error(fmt.Sprintf("Could not insert the final value of the variable after pruning, variable backup is stored in %s_PRUNE_BACKUP: %s", name, updateResp.Message)) - } - - // Delete the backup value - delErr := APIstub.DelState(fmt.Sprintf("%s_PRUNE_BACKUP", name)) - if delErr != nil { - return shim.Error(fmt.Sprintf("Could not delete backup value %s_PRUNE_BACKUP, this does not affect the ledger but should be removed manually", name)) + return shim.Error(fmt.Sprintf("Could not update the final value of the variable after pruning: %s", updateResp.Message)) } - return shim.Success([]byte(fmt.Sprintf("Successfully pruned variable %s, final value is %f, %d rows pruned", name, val, i))) + return shim.Success([]byte(fmt.Sprintf("Successfully pruned variable %s, final value is %f, %d rows pruned", args[0], finalVal, i))) } /** diff --git a/high-throughput/scripts/install-chaincode.sh b/high-throughput/scripts/install-chaincode.sh index 7817ae64cf..8bd230b0b6 100755 --- a/high-throughput/scripts/install-chaincode.sh +++ b/high-throughput/scripts/install-chaincode.sh @@ -9,25 +9,25 @@ export CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/pee export CORE_PEER_ADDRESS=peer0.org1.example.com:7051 export CORE_PEER_LOCALMSPID="Org1MSP" export CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt -peer chaincode install -n $CC_NAME -v $1 -p github.com/hyperledger/fabric/examples/chaincode/go +peer chaincode install -n $CC_NAME -v $1 -p github.com/hyperledger/fabric-samples/chaincode echo "========== Installing chaincode on peer1.org1 ==========" export CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp export CORE_PEER_ADDRESS=peer1.org1.example.com:7051 export CORE_PEER_LOCALMSPID="Org1MSP" export CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/peers/peer1.org1.example.com/tls/ca.crt -peer chaincode install -n $CC_NAME -v $1 -p github.com/hyperledger/fabric/examples/chaincode/go +peer chaincode install -n $CC_NAME -v $1 -p github.com/hyperledger/fabric-samples/chaincode echo "========== Installing chaincode on peer0.org2 ==========" export CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org2.example.com/users/Admin@org2.example.com/msp export CORE_PEER_ADDRESS=peer0.org2.example.com:7051 export CORE_PEER_LOCALMSPID="Org2MSP" export CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt -peer chaincode install -n $CC_NAME -v $1 -p github.com/hyperledger/fabric/examples/chaincode/go +peer chaincode install -n $CC_NAME -v $1 -p github.com/hyperledger/fabric-samples/chaincode echo "========== Installing chaincode on peer1.org2 ==========" export CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org2.example.com/users/Admin@org2.example.com/msp export CORE_PEER_ADDRESS=peer1.org2.example.com:7051 export CORE_PEER_LOCALMSPID="Org2MSP" export CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org2.example.com/peers/peer1.org2.example.com/tls/ca.crt -peer chaincode install -n $CC_NAME -v $1 -p github.com/hyperledger/fabric/examples/chaincode/go +peer chaincode install -n $CC_NAME -v $1 -p github.com/hyperledger/fabric-samples/chaincode diff --git a/high-throughput/scripts/prunefast-invoke.sh b/high-throughput/scripts/prune-invoke.sh similarity index 85% rename from high-throughput/scripts/prunefast-invoke.sh rename to high-throughput/scripts/prune-invoke.sh index bd7b168c5f..388f43649f 100755 --- a/high-throughput/scripts/prunefast-invoke.sh +++ b/high-throughput/scripts/prune-invoke.sh @@ -4,5 +4,5 @@ # SPDX-License-Identifier: Apache-2.0 # -peer chaincode invoke -o orderer.example.com:7050 --tls $CORE_PEER_TLS_ENABLED --cafile /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem -C $CHANNEL_NAME -n $CC_NAME -c '{"Args":["prunefast","'$1'"]}' +peer chaincode invoke -o orderer.example.com:7050 --tls $CORE_PEER_TLS_ENABLED --cafile /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem -C $CHANNEL_NAME -n $CC_NAME -c '{"Args":["prune","'$1'"]}' diff --git a/high-throughput/scripts/prunesafe-invoke.sh b/high-throughput/scripts/prunesafe-invoke.sh deleted file mode 100755 index e42019e8d8..0000000000 --- a/high-throughput/scripts/prunesafe-invoke.sh +++ /dev/null @@ -1,8 +0,0 @@ -# -# Copyright IBM Corp All Rights Reserved -# -# SPDX-License-Identifier: Apache-2.0 -# - -peer chaincode invoke -o orderer.example.com:7050 --tls $CORE_PEER_TLS_ENABLED --cafile /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem -C $CHANNEL_NAME -n $CC_NAME -c '{"Args":["prunesafe","'$1'"]}' -