R4R: New genesis workflow #2602

alessio · 2018-10-26T10:17:34Z

New workflow for chain initialization:

Run gaiad init to generate the default, empty genesis state.
Populate genesis.json accounts manually.
Generate a genesis tx with gaiad gentx
Collect all genesis txs and run gaiad collect-gentxs to get the new genesis.json
Run gaiad start.

Closes: #2596 #2615

Linked to github-issue with discussion and accepted design OR link to spec that describes this work.
Wrote tests
Updated relevant documentation (docs/)
Added entries in PENDING.md with issue #
rereviewed Files changed in the github PR explorer

For Admin Use:

Added appropriate labels to PR (ex. wip, ready-for-review, docs)
Reviewers Assigned
Squashed all commits, uses message "Merge pull request #XYZ: [title]" (coding standards)

codecov · 2018-10-28T22:36:12Z

Codecov Report

Merging #2602 into develop will decrease coverage by 0.54%.
The diff coverage is 15.45%.

@@             Coverage Diff             @@
##           develop    #2602      +/-   ##
===========================================
- Coverage     58.7%   58.16%   -0.55%     
===========================================
  Files          152      153       +1     
  Lines         9445     9476      +31     
===========================================
- Hits          5545     5512      -33     
- Misses        3530     3604      +74     
+ Partials       370      360      -10

rigelrozanski · 2018-10-30T14:55:53Z

localnet failure?

alessio · 2018-10-30T16:03:11Z

ERROR: insufficient fund for the delegation: 0 < 100
docker-compose up -d
Creating network "cosmossdk_localnet" with driver "bridge"

I'm investigating

alessio · 2018-10-30T17:16:38Z

Fixed @rigelrozanski

alessio · 2018-11-01T10:21:41Z

Docs will follow as soon as this is merged

alexanderbez

Thanks @alessio -- I left some initial minor feedback. I also want to actually run through this workflow and test it before giving an ACK 👍

cmd/gaia/app/genesis.go

alexanderbez · 2018-11-01T16:51:19Z

cmd/gaia/init/collect.go

+
+	genFile := config.GenesisFile()
+	// process genesis transactions, else create default genesis.json
+	var appGenTxs []auth.StdTx


Can we group these in a single var ( // ...) please?

cmd/gaia/init/collect.go

rigelrozanski · 2018-11-02T15:36:35Z

Talking with @alessio

General flow:

init creates default empty (init cmd)
populate accounts - edit the json manually (only address+coins)
each validator must generate a genesis-tx (gaiad gen-tx) (which is signed)
some distribute the gen-tx files, and add them to a specific folder (gaiad/config/gentx/)
Then there is some collect genesis-tx transaction???
gaiad start

I'd suggest that we group in the collect transaction with gaiad start when using a --collect flag

alessio · 2018-11-02T15:54:52Z

cmd/gaia/app/genesis.go

+		if acc.Coins.AmountOf(msg.Delegation.Denom).LT(msg.Delegation.Amount) {
+			err = fmt.Errorf("insufficient fund for the delegation: %s < %s",
+				acc.Coins.AmountOf(msg.Delegation.Denom), msg.Delegation.Amount)
+		}

 		// exclude itself from persistent peers
 		if msg.Description.Moniker != moniker {


Unnecessary as per @rigelrozanski

rigelrozanski

Good stuff - biggest comment was above about connecting collect and start command but that can be done as a separate PR in the future I think

Edit:
it was also be good to include the genesis workflow as you explained it to me (aka the above bullet points) in the docs somewhere as a part of this PR - even if it needs to change in further doc PRs

cmd/gaia/app/genesis.go

cmd/gaia/init/collect.go

cmd/gaia/init/gentx.go

cmd/gaia/init/init.go

cmd/gaia/init/testnet.go

* `gaiad init` is now used to generate an empty `genesis.json`. * Genesis accounts need to be populated manually before running `gaiad collect-gentxs`. * This should support starfish too, see #2615 for more info. Closes: #2596 #2615

…ante

rigelrozanski

waiting for @alexanderbez before merge - but looks good to me!

@rigelrozanski

* Back to 50 initially bonded * Add query deposits cmds * Update test * Update PENDING.md * Back to on-operation=false, update PENDING.md * Remove unnecessary change, clarify amt in sim_test.go * Cleanup, additional item in PENDING.md * Update PENDING.md Co-Authored-By: alessio <quadrispro@ubuntu.com> * Update client/keys/utils.go Co-Authored-By: alessio <quadrispro@ubuntu.com> * update depositer addr * Address @rigelrozanski comments * Make linter happy * Update PENDING.md * Remove shorthand * Make "multi" const * Store last total power as sdk.Int, not sdk.Dec * Merge PR cosmos#2553: Renamed msg.Name() and msg.Type() to msg.Type() and msg.Route() * Fix stupid mistake * s/number/weight/ * LastValidatorPower is also an Int * Merge PR cosmos#2462: Add swagger-ui for gov, stake and slashing * 'make format' * Move PENDING to CHANGELOG * Linkify changelog * Fix db write perm * Additional cleanup * Remove logs from prior debugging * Merge PR cosmos#2599 from cosmos/jae/dist_refactor * Rename Pool -> DelRewards; PoolCommission -> ValCommision * FeePool.Pool -> FeePool.ValPool * WithdrawalHeight->DelPoolWithdrawalHeight * OnValidatorBeginUnbonding * Caught the bug's tail * Update vi.FeePoolWithdrawalHeight upon bonding * Fix staking slashUnbondingDelegation bug; fixes simulator failure cosmos#9 * Merge PR cosmos#2597: Add distribution accum invariants * PENDING.md => CHANGELOG.md * Manually linkify * Manually fix some links * Docs fixes in progress while running through the release process * More docs fixes * '--voter' is no longer required * Rectify validator setup documentation * Merge PR cosmos#2596: Cmds for validator unbondings and redelegations * Make simulation use a transition matrix for block size This enables simulating periods of high load, and periods of low to no load. (low load because future ops will still terminate in that time frame) * address bez's comments * fix flags in docs, closes cosmos#2530 * Merge PR cosmos#2616: Block redelegations to the same validator * Merge PR cosmos#2623: Speedup simulator by switching to goleveldb Due to requiring app.Commit() at the moment, golevel db is significantly faster than a memdb * fix block offsets in printing simulation block number * Merge PR cosmos#2644: Simulation: Print last block when there is an error There was an off by one error in the log printing function previously * Merge PR cosmos#2642: Add todo diagrams * Various sign command improvements - Exit with error if the user is attempting to sign with a key whose address is not among those who are expected to sign the transaction. - Add --print-signature-only to output only the generated signature. * Check sanity of signatures and report errors when run with --print-sigs * Improve errors reporting * Improve online docs * Refresh PENDING.md * Find better name for --print-signature-only * Fix integration tests * Validate --name * Fix integration tests * s/--print-sigs/--validate-signatures/ * s/--sig-only/--signature-only/ * Docs updated * Update PENDING.md * Rename append, it's go builtin * Set success = false when it fails * Apply suggestions from bez * Nest switches * Fix rebase * Document what --validate-signatures does * perform minor doc and function cleanup * move typedef * Merge PR cosmos#2614: Configurable Bech32 prefix for SDK users * Merge PR cosmos#2643: AppendTag function usage error. append elements do not work * simulation: Make validator choice use validator set This also had to change the default seed, since with the previous one it actually got into a state where there were no validators left bonded, lol. This also changes Unbond msgs from failing with almost 100% probability to now only failing with 33% probability. Thus more of the state machine is getting tested! * Update changelog * Merge PR cosmos#2657: Fix config.js * Merge PR cosmos#2589: Update Vesting Spec * Merge PR cosmos#2656: Revert read-only leveldb database * Revert read-only leveldb database Waiting on a fix for syndtr/goleveldb#240. * Update client/keys/utils.go * Include DNS alt name in certificate Closes: cosmos#2664 * Gaialite signal handling is broken, repair it * Merge PR cosmos#2665: simulation: Remove header from Invariant This got introduced recently, but wasn't actually needed, hence the reversion * Merge PR cosmos#2653: Add benchmark for get and set account * Fix test * Refactor TrapSignal * Fix lint * enforcing @jaekwon mergemaster * added querier redelegation * added validatorDelegations querier endpoint * LCD and CLI * cli fixes * removed redelegation stuff * address other comments * rebased * addressed comments * Make the simulator create the new comission rate sensibly * Update to TM v0.26.0 - Part I (cosmos#2679) * Update to TM v0.26.0 * Bez/tm0.26 update pt 2 redux (cosmos#2684) * Update to TM v0.26.0 * Update TODOs * Proof and verification updates * Fix linting * Fix key path creation * Temporarily fix tendermint revision to make tests pass * Fix merge conflict bug; Update PENDING * New genesis workflow (cosmos#2602) New genesis workflow: * `gaiad init` is now used to generate an empty `genesis.json`. * Genesis accounts need to be populated manually before running `gaiad collect-gentxs`. * This should support starfish too, see cosmos#2615 for more info. * Closes: cosmos#2596 cosmos#2615 * Validate validator address and address against respective account ex ante * Fix local testnet failures * New genesis tests * Run make format * Add --pubkey flag * gaiad collect-gentxs takes no args * Simulation improvements (logging fix, random genesis parameters) (cosmos#2617) * Print out initial update on every block * Randomize simulation parameters * Randomize initial liveness weightings * Randomize genesis parameters * fixed power store invariant * IterateValidatorsBonded -> IterateBondedValidatorsByPower * WriteValidators uses IterateLastValidators rather than IterateBondedValidatorsByPower * fixed democoin interface Closes cosmos#2556 Closes cosmos#2396 Via cosmos#2671: closes cosmos#2669 closes cosmos#2670 closes cosmos#2620 Offshoot issues: cosmos#2618 cosmos#2619 cosmos#2620 cosmos#2661 * Fix simulation bugs; Incorprates cosmos#2676 from Sunny (cosmos#2677) * Fix simulation bugs; Incorprates cosmos#2676 from Sunny * Address review feedback; Update PENDING * 'make format' * Revert "enforcing @jaekwon mergemaster" This reverts commit 15c2093. * Update x/stake/client/rest/query.go Co-Authored-By: sunnya97 <sunnya97@gmail.com> * addressed fede's comment * Switch gov proposal-queues to use iterators (cosmos#2638) * switched gov proposals queue to use iterators * update gov spec * update proposal.Equal * Amino api change * switched proposalID to uint64 * renamed Gov Procedures to Params * s/ActiveProposalQueueProposalKey/KeyActiveProposalQueueProposal/g * numLatestProposals -> Limit * fixed staking invariant breakage because of gov deposits * Send deposits to DepositedCoinsAccAddr or BurnedDepositCoinsAccAddr * Add general merkle absence proof (also for empty substores) (cosmos#2685) * Fix coins.IsLT() impl (cosmos#2686) * Fix coins.IsLT() impl * Fix coin.IsLT() impl * Coins.IsLT -> Coins.IsAllLT etc * Update testnet to use canonical genesis time (cosmos#2692) * Update testnet to use canonical genesis time * Fix linting in genesis test * Do not allow nil values to be set in CacheKVStore (cosmos#2708) * Do not allow nil values to be set in CacheKVStore * Makefile OS compatibility update * Merge PR cosmos#2714: Add commission data to MsgCreateValidator signature bytes * PENDING => CHANGELOG * Linkify changelog * Cleanup bank keeper * whitespacing * rand utile ... * moving stuff around a bit, trying to get rid of types * reorganize more * rename ambig naming of queueOperations * minimizing indentation * fix some duplicate to get passing * Address style comments * Reorganize CLI command structure. Fixes cosmos#2575 * Fix missing flags issue * Address linting issues * Fix gobash CLI testing * Fix typo * Cross-compiling get_tools Makefile added * operations functions * assertAllInvarients changes, Operation reorg * mock tendermint * util cleanup * event stats object, more general cleanup * compiling * pending * Removed comment from Makefile as per bez's request * val comments * Address PR comments * Update cmd/gaia/cmd/gaiacli/main.go Co-Authored-By: jackzampolin <jack.zampolin@gmail.com> * PENDING * Fix state export/import, add to CI (cosmos#2690) * Update slashing import/export * More slashing.WriteGenesis * Add test import/export to CI * Store equality comparison. * Fix validator bond intra-tx counter * Set timeslices for unbonding validators * WriteGenesis => ExportGenesis * Delete validators from unbonding queue when re-bonded * Hook for validator deletion, fix staking genesis tests * Merge 0.26.0 back to develop (cosmos#2718) * PENDING => CHANGELOG * Linkify changelog * Merge PR cosmos#2716: Temporarily disable gaia lite insecure mode * TODO: need to update CHANGELOG w/ import-export PR cosmos#2690 * Update CHANGELOG/PENDING for straggling PR cosmos#2690 * Add small utility to add account to genesis.json after gaiad init * Update CHANGELOG.md * s/WriteGenesisFile/ExportGenesisFile/ * Update PENDING.md * Add --chain-id to testnet command * Address remaining comments from cosmos#2690 * Update PENDING.md * add back in PeriodicInvariant * Linter fix * Fix TimeoutCommit (cosmos#2743) * Fix TimeoutCommit to 5 seconds instead of whatever it was before which was too short. * Gaia-9000: Update to TM 0.26.1-rc2 (cosmos#2753) * Update to tm 0.26.1-rc2 to fix prometheus issue and node disconnect issue. * Gaia-9000: Update to TM 0.26.1-rc3 -- pex SeedMode fix * fix typo I think it might be a spelling mistake * Slight distribution spec cleanup * More cleanup * use defer * Use correct Bech32 prefix for show-address command (cosmos#2746) * Use consensus address bech32 prefix * Update show-address CLI description * Generate random moniker when missing * Update moniker prefix * Require moniker instead of generating a random one * update to tendermint v0.26.1 * Fix test coverage * Correctly set return code * Fix date to be cross platform * Merge PR cosmos#2752: Don't hardcode bondable denom * R4R: Fix unbonding command flow (cosmos#2727) * Fix required flag * Fix redelegation command * Add pending entry * update swagger.yaml * use newQuery...Params * Link to issue * Fix DiffKVStore * Address PR review * Working on stake import/export * Only apply validator set updates on initial genesis * Clarify comment * Fix failing test * add back in CLI command after rebase * Fix CLI tests * update to amino 0.14.1 * pending * R4R: Query Gov Params (cosmos#2576) * gov query params * Merge PR cosmos#2744: Fix Makefile targets dependencies * Fix Makefile targets dependencies * Remove unnecessary build deps from install targets * Create a rule for each tool * Don't dep test_lint on tools * Update docs/spec/distribution/overview.md Co-Authored-By: alexanderbez <alexanderbez@users.noreply.github.com> * Update docs/spec/distribution/overview.md Co-Authored-By: alexanderbez <alexanderbez@users.noreply.github.com> * Update docs/spec/distribution/overview.md Co-Authored-By: alexanderbez <alexanderbez@users.noreply.github.com> * Update overview.md * Documentation Structure Change and Cleanup (cosmos#2808) * Update docs/sdk/clients.md * organize ADR directory like tendermint * docs: move spec-proposals into spec/ * remove lotion, moved to website repo * move getting-started to cosmos-hub, and voyager to website * docs: move lite/ into clients/lite/ * move introduction/ content to website repo * move resources/ content to website repo * mv sdk/clients.md to clients/clients.md * mv validators to cosmos-hub/validators * move deprecated sdk/ content to _attic * sdk/modules.md is duplicate with modules/README.md * consolidate remianing sdk/ files into a single sdk.md * move examples/ to docs/examples/ * mv docs/cosmos-hub to docs/gaia * Add keys/accounts section to localnet docs * Bring back banner (cosmos#2814) * Build docs in CircleCI (cosmos#2810) * error checking the API call * added docs build trigger to circleci job * Update contributing.md with new merge policy (cosmos#2789) * Update contribuiting.md with new merge policy * deleted obsolete file (cosmos#2817)

alessio added the wip label Oct 26, 2018

alessio requested review from cwgoes, ebuchman and rigelrozanski as code owners October 26, 2018 10:17

alessio requested a review from jaekwon October 26, 2018 10:17

alessio force-pushed the alessio/2596-new-genesis-workflow branch from b78fa57 to 37128e7 Compare October 28, 2018 23:11

alessio changed the title ~~WIP: Start working on the new genesis workflow~~ R4R: Start working on the new genesis workflow Oct 29, 2018

alessio added ready-for-review and removed wip labels Oct 29, 2018

alessio requested a review from ValarDragon October 29, 2018 18:10

alessio force-pushed the alessio/2596-new-genesis-workflow branch from fb57ccc to 02c147b Compare October 30, 2018 10:24

faboweb mentioned this pull request Oct 30, 2018

Gaiad init posts output to stderr #2628

Closed

4 tasks

alessio requested a review from zmanian October 31, 2018 14:17

alessio force-pushed the alessio/2596-new-genesis-workflow branch from 22f57fe to 03ffd94 Compare November 1, 2018 10:20

alessio changed the title ~~R4R: Start working on the new genesis workflow~~ R4R: New genesis workflow Nov 1, 2018

alexanderbez suggested changes Nov 1, 2018

View reviewed changes

alessio commented Nov 2, 2018

View reviewed changes

rigelrozanski reviewed Nov 2, 2018

View reviewed changes

Alessio Treglia added 6 commits November 2, 2018 18:02

New genesis workflow

34a64c3

* `gaiad init` is now used to generate an empty `genesis.json`. * Genesis accounts need to be populated manually before running `gaiad collect-gentxs`. * This should support starfish too, see #2615 for more info. Closes: #2596 #2615

Validate validator address and address against respective account ex …

2a611e0

…ante

Fix local testnet failures

1591409

New genesis tests

c728863

Add --pubkey flag

f7d8ab6

Run make format

fbfdf12

rigelrozanski and others added 17 commits November 2, 2018 18:02

minor updates during review (comments, names)

da5d3a2

Remove commented

61da238

Remove validators from comment

88ba6d2

Add newline

191e85e

Shorten line

d2fa438

check the type and return an error if it's not MsgCreateValidator

13add31

Use constant

dbaf13f

Renaming

d72cb6b

ACK Rigel comment

16bc4b6

Return explicit variables

8dbd6c3

s/DefaultGenesisState/NewDefaultGenesisState/

98a5218

Use explicit variables

72a4dae

More compact

3e9c5af

Lowercase comment

90531d4

Use bondDenom

814e664

Delete old code

ac22aad

gaiad collect-gentxs takes no args

7f1f422

alessio force-pushed the alessio/2596-new-genesis-workflow branch from 3994aca to 7f1f422 Compare November 2, 2018 18:03

rigelrozanski approved these changes Nov 2, 2018

View reviewed changes

Update PENDING

a0c6f0b

jaekwon merged commit c20fcbf into develop Nov 5, 2018

alessio mentioned this pull request Nov 5, 2018

"gaiad gentx" might need to support additional argument for use in starfish #2615

Closed

alessio deleted the alessio/2596-new-genesis-workflow branch November 28, 2018 18:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

R4R: New genesis workflow #2602

R4R: New genesis workflow #2602

alessio commented Oct 26, 2018 •

edited

Loading

codecov bot commented Oct 28, 2018 •

edited

Loading

rigelrozanski commented Oct 30, 2018

alessio commented Oct 30, 2018 •

edited

Loading

alessio commented Oct 30, 2018

alessio commented Nov 1, 2018

alexanderbez left a comment

alexanderbez Nov 1, 2018

rigelrozanski commented Nov 2, 2018 •

edited

Loading

alessio Nov 2, 2018

rigelrozanski left a comment •

edited

Loading

rigelrozanski left a comment

R4R: New genesis workflow #2602

R4R: New genesis workflow #2602

Conversation

alessio commented Oct 26, 2018 • edited Loading

codecov bot commented Oct 28, 2018 • edited Loading

Codecov Report

rigelrozanski commented Oct 30, 2018

alessio commented Oct 30, 2018 • edited Loading

alessio commented Oct 30, 2018

alessio commented Nov 1, 2018

alexanderbez left a comment

Choose a reason for hiding this comment

alexanderbez Nov 1, 2018

Choose a reason for hiding this comment

rigelrozanski commented Nov 2, 2018 • edited Loading

alessio Nov 2, 2018

Choose a reason for hiding this comment

rigelrozanski left a comment • edited Loading

Choose a reason for hiding this comment

rigelrozanski left a comment

Choose a reason for hiding this comment

alessio commented Oct 26, 2018 •

edited

Loading

codecov bot commented Oct 28, 2018 •

edited

Loading

alessio commented Oct 30, 2018 •

edited

Loading

rigelrozanski commented Nov 2, 2018 •

edited

Loading

rigelrozanski left a comment •

edited

Loading