IPC: Implements mir-validator, mir consensus and decouples storage #9492
Conversation
Co-authored-by: Raúl Kripalani <raul@protocol.ai>
This fiexes issue #8771 (filecoin-project/lotus#8771)
| // Note that the private key used to produce the signature cannot be set ("registered") through this interface. | ||
| // Storing and using the private key is completely implementation-dependent. | ||
| func (c *CryptoManager) Sign(data [][]byte) ([]byte, error) { | ||
| signature, err := c.api.WalletSign(context.Background(), c.key, hash(data)) |
There was a problem hiding this comment.
Use domain separation tags if at all possible
There was a problem hiding this comment.
You mean for the signature?
There was a problem hiding this comment.
@magik6k Could you provide an example or the rules to understand how to separate domains properly not to create collisions?
| import ( | ||
| ipfslogging "github.com/ipfs/go-log/v2" | ||
|
|
||
| mirlogging "github.com/filecoin-project/mir/pkg/logging" |
There was a problem hiding this comment.
Why not move mir to use https://github.com/ipfs/go-log, like every PL Go project?
There was a problem hiding this comment.
@matejpavlovic, @dnkolegov to answer this one.
There was a problem hiding this comment.
Mir uses go-log. That's just a wrapper on top of it.
| "github.com/filecoin-project/mir/pkg/pb/checkpointpb" | ||
| "github.com/filecoin-project/mir/pkg/pb/commonpb" | ||
| "github.com/filecoin-project/mir/pkg/pb/requestpb" |
There was a problem hiding this comment.
Why is mir using protobuf when the entire Filecoin protocol is cbor-based? Protobufs are going to be a pain to deal with in IPLD.
There was a problem hiding this comment.
Mir was @matejpavlovic's project before joining PL. The project used protobuf already and this is why we didn't include cbor in the integration. I am pro-cbor, but again, I'll defer this one to @matejpavlovic.
| } | ||
|
|
||
| // GetValidatorsFromCfg gets the membership config from a file. | ||
| func GetValidatorsFromCfg(config string) (*ValidatorSet, error) { |
There was a problem hiding this comment.
This probably should come from an embedded file in build/
There was a problem hiding this comment.
(Or I guess in node repo because we write this file? Maybe in metadata datastore)
There was a problem hiding this comment.
We didn't include it in build/ because in the future the validator set will be dynamic (and handled on-chain), but I agree that we could use the datastore instead of a config file). We'll look into it. 🙏
| return "", nil | ||
| } | ||
|
|
||
| func (bft *Mir) ValidateBlock(ctx context.Context, b *types.FullBlock) (err error) { |
There was a problem hiding this comment.
Where do we verify that a block coming from the network is legit?
There was a problem hiding this comment.
This PR adds only support for Mir validators. Validators can only receive blocks through the orderer layer, and they don't broadcast blocks to the network. Support for full-nodes is introduced in the next PR (maybe it would have made sense to merge both, but it was getting huge).
The relevant code for block validation can be found here: https://github.com/consensus-shipyard/lotus/blob/510c6dfc77485aa463cde07a9f9bccee86addc87/chain/consensus/mir/consensus.go#L106
Sorry if we made this a mess, happy to merge PRs and make it a single on, and/or walk you through the code if it helps 🙏
| �e� ��c�?�*b pi���8[�z� ���Z�SM���cȠ�3��T� | ||
| Ǝ9�z�5�4 |
There was a problem hiding this comment.
We use it to run a local mir network fast and do some manual testing. Will remove it before merging. Thanks.
|
Closing for now to reduce noise, I don't think there is a plan to merge Mir into Lotus at the moment. |
Related Issues
Proposed Changes
This PR integrates the Mir framework and implements a pBFT consensus algorithm over Mir. Mir is implemented as an alternative consensus algorithm for Lotus. In order to run Lotus with Mir, a new build parameter
build.Consensushas been provided.This PR includes:
chain/consensus/mirof Mir as a new consensus interface.mir-validatorprocess that implements all the mining and block producing logic for Mir.make mir-validatorcan be used to compile the process. This process is responsible for ordering new transactions and proposing new blocks to the network.spacenetcompile parameters (seebuild/params_spacenet.go) and a newbuild.Consensusflag used to determine the type of consensus to use. Using the Mir consensus disables the block production andwinningPoStprocess fromlotus-minerand forces the use of fake randomness forwindowPoSt.lotus-minercan still be used (and run) in a Mir Lotus network for storage-specific features (on boarding storage, starting storage and retrieval deals, markets, etc.).What's next?
We currently don't support the ability for other full nodes in the network to sync with a Mir network. Only validators running the consensus algorithm get new blocks. The next PR will include the syncing protocol (implementing consensus-shipyard/lotus#4)