Skip to content

Commit

Permalink
[ASHF-48] Update base (#24)
Browse files Browse the repository at this point in the history
* [FAB-17819] Discovery returns user friendly errors

Currently, the discovery service filters out peers that don't have the chaincode installed
early on in the computation, and as a result - the service cannot distinguish from a case
where there are not enough alive peers to satisfy the endorsement policy, or that there are
enough peers but the chaincode is not installed on enough of them.

This change set defers the chaincode filtering to the end of the computation, so
the layouts and peer group mapping is creating without taking into account if the peers
have the chaincode installed on them, and if there is no layout that can be satisfied
without taking into account the chaincodes - the error that is returned now
is "no peer combination can satisfy the endorsement policy",
instead of "cannot satisfy any principal combination".

Afterwards, the layouts are being inspected once again, and then the layouts
that cannot be satisfied are filtered out, when the error returned
when no layout can be satisfied is now: "required chaincodes are not installed on sufficient peers".

Change-Id: I74eb29b30aec1a87842d220414c73872cdbc8304
Signed-off-by: yacovm <yacovm@il.ibm.com>

* Fix docker network leak from RAFT integration test (hyperledger#1203)

Signed-off-by: Matthew Sykes <matthew.sykes@gmail.com>

* [FAB-17786] Update upgrade_dbs peer command to drop state couchdb (hyperledger#1187)

The existing upgrade_dbs command does not automatically drop state couchdbs
and therefore a separate step is required to drop couchdbs, This PR updates
the command to automatically drop state couchdbs.

In addition, it checks upgrade eligibility before upgrade so that it will not
drop databases if it is already the expected format.

Signed-off-by: Wenjian Qiao <wenjianq@gmail.com>

* [FAB-17774] support orderer restart without genesis block (hyperledger#1197)

* adding support for orderer restart without genesis block.

Signed-off-by: Chongxin Luo <Chongxin.Luo@ibm.com>

* Remove interface for block storage

- Removed the interface for better code navigation, as There is only a
 single implementation for block storage
- Merged the remaining code in the package blkstorgae with the
implementation in the package fsblkstorage and used the name
blkstorgae for the final package
- Moved the internal single proto message with in same package

Signed-off-by: manish <manish.sethi@gmail.com>

* [LEDGER] Move UpdatesBytesBuilder to txmgr pkg (hyperledger#1209)

- mv updateBatch bytes constructor to txmgr pkg
Currently, we create the bytes representation of updateBatch
to compute the hash of updateBatch which would be included in
the block along with the validation results. The utility for
converting a updateBatch to a deterministic bytes is present
in the privacyenabledstate pkg. However, this utility is
used only by the txmgr and hence, we move the utility
function to txmgr pkg from privacyenabledstate pkg.

- rename proto messages
In the updateBatch proto, we have defined proto messages such
as KVWriteProto and KVWriteBatchProto. As we shouldn't append
the keyword proto to messages, we rename KVWriteProto to KVWrite
and KVWriteBatchProto to Updates.

- rename function names
The function name such as buildForKeys(), buildForColls() are not
very explicit in what they do. When the buildForColls() calls
buildForKeys(), it adds even more confusion as the first-level
function, i.e., deterministicBytesForPubAndHashUpdates() is also
calling buildForKeys(). Hence, we have used the following function
names instead:
(1) genKVsFromNsUpdates()
(2) genKVsFromCollsUpdates()
(3) genKVs()

FAB-17830

Signed-off-by: senthil <cendhu@gmail.com>

* Update grpc-go to v1.29.1 (hyperledger#1213)

Signed-off-by: Gari Singh <gari.r.singh@gmail.com>

* [FAB-17831] Use generic constant/var names in dataformat.go (hyperledger#1212)

Currently Version1x and Version20 are defined in dataformat.go.
They should be renamed to more generic names such as
PreviousFormat and CurrentFormat.

The format values will change only when the data format is changed
in ledger. If a Fabric version does not introduce a new data format,
CurrentFormat will remain the same as the latest format prior to
the Fabric version.

Signed-off-by: Wenjian Qiao <wenjianq@gmail.com>

* [LEDGER] rm interface from pvtdatastorage pkg (hyperledger#1217)

rm pvtdatastorage interfaces

As we have only one implementation of the pvtdatastore and not
planning to any a new one, we can safely remove the interface.
This also helps in code navigation and to avoid type casting
such as s.(*store) in the test.

FAB-17843

Signed-off-by: senthil <cendhu@gmail.com>

* FAB-15710 Ch.Part.API: orderer config & hook into http server (hyperledger#1218)

- Expose a configuration object for channel participation at
  the top level (like Metrics). Even though the channel
  participation API shares the same endpoint and TLS config
  with operations, placing it at the top level will allow us
  to separate these APIs to different endpoints in the future
  without changing the config structure.

- Extend operations.System to be able to register a handler
  for additional APIs.

- Implement a skeleton handler for the channel participation API.

- Register the skeleton handler to the http server in
  operations.System during the server boot sequence.

Signed-off-by: Yoav Tock <tock@il.ibm.com>
Change-Id: I5cf15dffa29985cba60e5aaf31d189e755a3a1ef

* Update the vagrant dev environment

- Remove GOPATH in favor of modules
- Move to Ubuntu 20.04
- Remove docker-compose as it's unnecessary for build and test

Signed-off-by: Matthew Sykes <sykesmat@us.ibm.com>

* Add function in blockstore to export TxIDs

FAB-17837

Signed-off-by: manish <manish.sethi@gmail.com>

* reduce #arguments in a few kvledger methods (hyperledger#1210)

As the number of arguments passed to newKVLedger(),
initTxMgr() is quite high, we reduce it by introducing
initializer struct.

FAB-17683

Signed-off-by: senthil <cendhu@gmail.com>

* Clarify the deliver access denied message (hyperledger#1224)

There are two scenarios where a deliver client could receive a
'FORBIDDEN' result when requesting blocks.  Either the client was not
authorized to connect to the channel initially, or, the client's
access was revoked after a successful connection by some later
configuration block.  In both cases, we log an identical error message
that "Client authorization revoked" when in fact, for the first case,
the client may never have had access, so claiming it was revoked is
misleading.

Signed-off-by: Jason Yellick <jyellick@us.ibm.com>

* Validate TLS certs during raft consenter addition (hyperledger#1223)

* raft membership_test.go cleanup

- For negative tests, test that expected error message is returned.
- For positive tests, switch the order so expected/actual match the
expected usage of "testify/require" package.

Signed-off-by: Will Lahti <wtlahti@us.ibm.com>

* Validate TLS certs during raft consenter addition

FAB-17733 #done

Signed-off-by: Will Lahti <wtlahti@us.ibm.com>

* [FAB-17640] Remove pkg/configtx and import as hyperledger/fabric-config

Signed-off-by: Danny Cao <dcao@us.ibm.com>

* Update CONTRIBUTING guide

Removed reference to Gerrit

Signed-off-by: Ry Jones <ry@linux.com>

Co-authored-by: yacovm <yacovm@il.ibm.com>
Co-authored-by: Matthew Sykes <sykesmat@us.ibm.com>
Co-authored-by: wen <wenjianq@gmail.com>
Co-authored-by: Dereck <Chongxin.Luo@ibm.com>
Co-authored-by: manish <manish.sethi@gmail.com>
Co-authored-by: Senthil Nathan N <cendhu@users.noreply.github.com>
Co-authored-by: Gari Singh <gari.r.singh@gmail.com>
Co-authored-by: Yoav Tock <tock@il.ibm.com>
Co-authored-by: Jason Yellick <jyellick@us.ibm.com>
Co-authored-by: Will Lahti <wtlahti@us.ibm.com>
Co-authored-by: Danny Cao <dcao@us.ibm.com>
Co-authored-by: Ry Jones <ry@linux.com>
  • Loading branch information
13 people authored May 10, 2020
1 parent a233313 commit f877875
Show file tree
Hide file tree
Showing 43 changed files with 2,436 additions and 12,436 deletions.
12 changes: 12 additions & 0 deletions common/crypto/tlsgen/ca.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@ type CA interface {
// CertBytes returns the certificate of the CA in PEM encoding
CertBytes() []byte

NewIntermediateCA() (CA, error)

// newCertKeyPair returns a certificate and private key pair and nil,
// or nil, error in case of failure
// The certificate is signed by the CA and is used for TLS client authentication
Expand All @@ -55,6 +57,16 @@ func NewCA() (CA, error) {
return c, nil
}

func (c *ca) NewIntermediateCA() (CA, error) {
intermediateCA := &ca{}
var err error
intermediateCA.caCert, err = newCertKeyPair(true, false, "", c.caCert.Signer, c.caCert.TLSCert)
if err != nil {
return nil, err
}
return intermediateCA, nil
}

// CertBytes returns the certificate of the CA in PEM encoding
func (c *ca) CertBytes() []byte {
return c.caCert.Cert
Expand Down
2 changes: 1 addition & 1 deletion common/deliver/deliver.go
Original file line number Diff line number Diff line change
Expand Up @@ -249,7 +249,7 @@ func (h *Handler) deliverBlocks(ctx context.Context, srv *Server, envelope *cb.E
}

if err := accessControl.Evaluate(); err != nil {
logger.Warningf("[channel: %s] Client authorization revoked for deliver request from %s: %s", chdr.ChannelId, addr, err)
logger.Warningf("[channel: %s] Client %s is not authorized: %s", chdr.ChannelId, addr, err)
return cb.Status_FORBIDDEN, nil
}

Expand Down
10 changes: 0 additions & 10 deletions docs/source/CONTRIBUTING.rst
Original file line number Diff line number Diff line change
@@ -1,13 +1,3 @@
.. note:: Users who are migrating from Gerrit to GitHub: You can follow simple
Git workflows to move your development from Gerrit to GitHub. After
forking the Fabric repo, simply push the branches you want to save from
your current Gerrit-based local repo to your remote forked repository.
Once you've pushed the changes you want to save, simply delete your
local Gerrit-based repository and clone your fork.

For a basic Git workflow recommendation please see our doc at
:doc:`github/github`.

Contributions Welcome!
======================

Expand Down
3 changes: 2 additions & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,9 @@ require (
github.com/hashicorp/go-version v1.2.0
github.com/hyperledger/fabric-amcl v0.0.0-20200128223036-d1aa2665426a
github.com/hyperledger/fabric-chaincode-go v0.0.0-20200128192331-2d899240a7ed
github.com/hyperledger/fabric-config v0.0.0-20200507183426-92ce5d82acee
github.com/hyperledger/fabric-lib-go v1.0.0
github.com/hyperledger/fabric-protos-go v0.0.0-20200326212758-d7d9b8e1fcde
github.com/hyperledger/fabric-protos-go v0.0.0-20200424173316-dd554ba3746e
github.com/konsorten/go-windows-terminal-sequences v1.0.2 // indirect
github.com/kr/pretty v0.2.0
github.com/magiconair/properties v1.8.1 // indirect
Expand Down
6 changes: 4 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -132,11 +132,13 @@ github.com/hyperledger/fabric-amcl v0.0.0-20200128223036-d1aa2665426a h1:HgdNn3U
github.com/hyperledger/fabric-amcl v0.0.0-20200128223036-d1aa2665426a/go.mod h1:X+DIyUsaTmalOpmpQfIvFZjKHQedrURQ5t4YqquX7lE=
github.com/hyperledger/fabric-chaincode-go v0.0.0-20200128192331-2d899240a7ed h1:VNnrD/ilIUO9DDHQP/uioYSy1309rYy0Z1jf3GLNRIc=
github.com/hyperledger/fabric-chaincode-go v0.0.0-20200128192331-2d899240a7ed/go.mod h1:N7H3sA7Tx4k/YzFq7U0EPdqJtqvM4Kild0JoCc7C0Dc=
github.com/hyperledger/fabric-config v0.0.0-20200507183426-92ce5d82acee h1:boJpvFbqY+z2C5hx5eZIUnbIK3PpUlHCHTDkJ/isoXg=
github.com/hyperledger/fabric-config v0.0.0-20200507183426-92ce5d82acee/go.mod h1:aeDZ0moG/qKvwLjddcqYr8+58/oNaJy3HE0tI01546c=
github.com/hyperledger/fabric-lib-go v1.0.0 h1:UL1w7c9LvHZUSkIvHTDGklxFv2kTeva1QI2emOVc324=
github.com/hyperledger/fabric-lib-go v1.0.0/go.mod h1:H362nMlunurmHwkYqR5uHL2UDWbQdbfz74n8kbCFsqc=
github.com/hyperledger/fabric-protos-go v0.0.0-20190919234611-2a87503ac7c9/go.mod h1:xVYTjK4DtZRBxZ2D9aE4y6AbLaPwue2o/criQyQbVD0=
github.com/hyperledger/fabric-protos-go v0.0.0-20200326212758-d7d9b8e1fcde h1:mRimjHLoK6kPW9EJv+//uZZKJgnAIFPYHQxsjlCVwqQ=
github.com/hyperledger/fabric-protos-go v0.0.0-20200326212758-d7d9b8e1fcde/go.mod h1:xVYTjK4DtZRBxZ2D9aE4y6AbLaPwue2o/criQyQbVD0=
github.com/hyperledger/fabric-protos-go v0.0.0-20200424173316-dd554ba3746e h1:9PS5iezHk/j7XriSlNuSQILyCOfcZ9wZ3/PiucmSE8E=
github.com/hyperledger/fabric-protos-go v0.0.0-20200424173316-dd554ba3746e/go.mod h1:xVYTjK4DtZRBxZ2D9aE4y6AbLaPwue2o/criQyQbVD0=
github.com/ijc/Gotty v0.0.0-20170406111628-a8b993ba6abd h1:anPrsicrIi2ColgWTVPk+TrN42hJIWlfPHSBP9S0ZkM=
github.com/ijc/Gotty v0.0.0-20170406111628-a8b993ba6abd/go.mod h1:3LVOLeyx9XVvwPgrt2be44XgSqndprz1G18rSk8KD84=
github.com/inconshreveable/mousetrap v1.0.0 h1:Z8tu5sraLXCXIcARxBp/8cbvlwVa7Z1NHg9XEKhtSvM=
Expand Down
2 changes: 1 addition & 1 deletion integration/configtx/configtx_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@ import (

docker "github.com/fsouza/go-dockerclient"
"github.com/golang/protobuf/proto"
"github.com/hyperledger/fabric-config/configtx"
"github.com/hyperledger/fabric/integration/nwo"
"github.com/hyperledger/fabric/integration/nwo/commands"
"github.com/hyperledger/fabric/pkg/configtx"
"github.com/tedsuo/ifrit"

. "github.com/onsi/ginkgo"
Expand Down
87 changes: 64 additions & 23 deletions integration/raft/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ import (
"github.com/hyperledger/fabric-protos-go/msp"
protosorderer "github.com/hyperledger/fabric-protos-go/orderer"
"github.com/hyperledger/fabric-protos-go/orderer/etcdraft"
"github.com/hyperledger/fabric/common/crypto/tlsgen"
"github.com/hyperledger/fabric/integration/nwo"
"github.com/hyperledger/fabric/integration/nwo/commands"
"github.com/hyperledger/fabric/internal/configtxgen/encoder"
Expand Down Expand Up @@ -56,7 +57,7 @@ var _ = Describe("EndToEnd reconfiguration and onboarding", func() {
ordererProcesses = nil

var err error
testDir, err = ioutil.TempDir("", "e2e-etcfraft_reconfig")
testDir, err = ioutil.TempDir("", "e2e-etcdraft_reconfig")
Expect(err).NotTo(HaveOccurred())

client, err = docker.NewClientFromEnv()
Expand Down Expand Up @@ -303,9 +304,10 @@ var _ = Describe("EndToEnd reconfiguration and onboarding", func() {
By("Waiting for a leader to be re-elected")
findLeader(ordererRunners)

// In the next part of the test we're going to bring up a third node with a different TLS root CA
// and we're going to add the TLS root CA *after* we add it to the channel, to ensure
// that we can dynamically update TLS root CAs in Raft while membership stays the same.
// In the next part of the test we're going to bring up a third node
// with a different TLS root CA. We're then going to remove the TLS
// root CA and restart the orderer, to ensure that we can dynamically
// update TLS root CAs in Raft while membership stays the same.

By("Creating configuration for a third orderer with a different TLS root CA")
orderer3 := &nwo.Orderer{
Expand Down Expand Up @@ -369,20 +371,12 @@ var _ = Describe("EndToEnd reconfiguration and onboarding", func() {
Expect(err).NotTo(HaveOccurred())
}

By("Adding the third orderer to the channel")
addConsenter(network, peer, orderer, "systemchannel", etcdraft.Consenter{
ServerTlsCert: thirdOrdererCertificate,
ClientTlsCert: thirdOrdererCertificate,
Host: "127.0.0.1",
Port: uint32(network.OrdererPort(orderer3, nwo.ClusterPort)),
})

By("Obtaining the last config block from the orderer once more to update the bootstrap file")
configBlock = nwo.GetConfigBlock(network, peer, orderer, "systemchannel")
err = ioutil.WriteFile(filepath.Join(testDir, "systemchannel_block.pb"), protoutil.MarshalOrPanic(configBlock), 0644)
Expect(err).NotTo(HaveOccurred())

By("Launching the third orderer")
By("Launching orderer3")
launch(orderer3)

By("Expanding the TLS root CA certificates")
Expand All @@ -391,20 +385,67 @@ var _ = Describe("EndToEnd reconfiguration and onboarding", func() {
return config
})

By("Adding orderer3 to the channel")
addConsenter(network, peer, orderer, "systemchannel", etcdraft.Consenter{
ServerTlsCert: thirdOrdererCertificate,
ClientTlsCert: thirdOrdererCertificate,
Host: "127.0.0.1",
Port: uint32(network.OrdererPort(orderer3, nwo.ClusterPort)),
})

By("Waiting for orderer3 to see the leader")
leader := findLeader([]*ginkgomon.Runner{ordererRunners[2]})
leaderIndex := leader - 1
findLeader([]*ginkgomon.Runner{ordererRunners[2]})

fmt.Fprint(GinkgoWriter, "Killing the leader", leader)
ordererProcesses[leaderIndex].Signal(syscall.SIGTERM)
Eventually(ordererProcesses[leaderIndex].Wait(), network.EventuallyTimeout).Should(Receive())
By("Removing orderer3's TLS root CA certificate")
nwo.UpdateOrdererMSP(network, peer, orderer, "systemchannel", "OrdererOrg", func(config msp.FabricMSPConfig) msp.FabricMSPConfig {
config.TlsRootCerts = config.TlsRootCerts[:len(config.TlsRootCerts)-1]
return config
})

By("Ensuring orderer3 detects leader loss")
leaderLoss := fmt.Sprintf("Raft leader changed: %d -> 0", leader)
Eventually(ordererRunners[2].Err(), network.EventuallyTimeout, time.Second).Should(gbytes.Say(leaderLoss))
By("Killing orderer3")
o3Proc := ordererProcesses[2]
o3Proc.Signal(syscall.SIGKILL)
Eventually(o3Proc.Wait(), network.EventuallyTimeout).Should(Receive(MatchError("exit status 137")))

By("Restarting orderer3")
o3Runner := network.OrdererRunner(orderer3)
ordererRunners[2] = o3Runner
o3Proc = ifrit.Invoke(o3Runner)
Eventually(o3Proc.Ready(), network.EventuallyTimeout).Should(BeClosed())
ordererProcesses[2] = o3Proc

By("Ensuring TLS handshakes fail with the other orderers")
for i, oRunner := range ordererRunners {
if i < 2 {
Eventually(oRunner.Err(), network.EventuallyTimeout).Should(gbytes.Say("TLS handshake failed with error tls: client didn't provide a certificate"))
continue
}
Eventually(oRunner.Err(), network.EventuallyTimeout).Should(gbytes.Say("TLS handshake failed with error remote error: tls: bad certificate"))
Eventually(oRunner.Err(), network.EventuallyTimeout).Should(gbytes.Say("Suspecting our own eviction from the channel"))
}

By("Waiting for the leader to be re-elected")
findLeader([]*ginkgomon.Runner{ordererRunners[2]})
By("Attemping to add a consenter with invalid certs")
// create new certs that are not in the channel config
ca, err := tlsgen.NewCA()
Expect(err).NotTo(HaveOccurred())
client, err := ca.NewClientCertKeyPair()
Expect(err).NotTo(HaveOccurred())

current, updated := consenterAdder(
network,
peer,
orderer,
"systemchannel",
etcdraft.Consenter{
ServerTlsCert: client.Cert,
ClientTlsCert: client.Cert,
Host: "127.0.0.1",
Port: uint32(network.OrdererPort(orderer3, nwo.ListenPort)),
},
)
sess = nwo.UpdateOrdererConfigSession(network, orderer, network.SystemChannel.Name, current, updated, peer, orderer)
Eventually(sess, network.EventuallyTimeout).Should(gexec.Exit(1))
Expect(sess.Err).To(gbytes.Say(fmt.Sprintf("BAD_REQUEST -- error applying config update to existing channel 'systemchannel': consensus metadata update for channel config update is invalid: verifying tls client cert with serial number %d: x509: certificate signed by unknown authority", client.TLSCert.SerialNumber)))
})
})

Expand Down
4 changes: 2 additions & 2 deletions orderer/consensus/etcdraft/chain.go
Original file line number Diff line number Diff line change
Expand Up @@ -942,7 +942,7 @@ func (c *Chain) detectConfChange(block *common.Block) *MembershipChanges {
c.sizeLimit = configMetadata.Options.SnapshotIntervalSize
}

changes, err := ComputeMembershipChanges(c.opts.BlockMetadata, c.opts.Consenters, configMetadata.Consenters)
changes, err := ComputeMembershipChanges(c.opts.BlockMetadata, c.opts.Consenters, configMetadata.Consenters, c.support.SharedConfig())
if err != nil {
c.logger.Panicf("illegal configuration change detected: %s", err)
}
Expand Down Expand Up @@ -1306,7 +1306,7 @@ func (c *Chain) ValidateConsensusMetadata(oldMetadataBytes, newMetadataBytes []b
// create the dummy parameters for ComputeMembershipChanges
dummyOldBlockMetadata, _ := ReadBlockMetadata(nil, oldMetadata)
dummyOldConsentersMap := CreateConsentersMap(dummyOldBlockMetadata, oldMetadata)
changes, err := ComputeMembershipChanges(dummyOldBlockMetadata, dummyOldConsentersMap, newMetadata.Consenters)
changes, err := ComputeMembershipChanges(dummyOldBlockMetadata, dummyOldConsentersMap, newMetadata.Consenters, c.support.SharedConfig())
if err != nil {
return err
}
Expand Down
Loading

0 comments on commit f877875

Please sign in to comment.