WIP Exploring x/distribution cleanup #2599

jaekwon · 2018-10-26T00:52:03Z

Just pushing my changes for anyone to track my progress.

codecov · 2018-10-26T00:55:32Z

Codecov Report

Merging #2599 into develop will decrease coverage by 0.04%.
The diff coverage is 46.29%.

@@             Coverage Diff             @@
##           develop    #2599      +/-   ##
===========================================
- Coverage    58.85%   58.81%   -0.05%     
===========================================
  Files          152      152              
  Lines         9411     9420       +9     
===========================================
+ Hits          5539     5540       +1     
- Misses        3503     3510       +7     
- Partials       369      370       +1

rigelrozanski · 2018-10-26T01:41:00Z

Makefile

@@ -169,7 +169,7 @@ test_sim_gaia_nondeterminism:

 test_sim_gaia_fast:
 	@echo "Running quick Gaia simulation. This may take several minutes..."
-	@go test ./cmd/gaia/app -run TestFullGaiaSimulation -SimulationEnabled=true -SimulationNumBlocks=400 -SimulationBlockSize=200 -SimulationCommit=true -v -timeout 24h
+	@go test ./cmd/gaia/app -run TestFullGaiaSimulation -SimulationEnabled=true -SimulationNumBlocks=400 -SimulationBlockSize=200 -SimulationCommit=true -SimulationSeed=1 -v -timeout 24h


We should probably revert this

rigelrozanski · 2018-10-26T01:41:59Z

x/distribution/keeper/allocation.go

@@ -27,8 +27,8 @@ func (k Keeper) AllocateTokens(ctx sdk.Context, percentVotes sdk.Dec, proposer s
 	// apply commission
 	commission := proposerReward.MulDec(proposerValidator.GetCommission())
 	remaining := proposerReward.Minus(commission)
-	proposerDist.PoolCommission = proposerDist.PoolCommission.Plus(commission)
-	proposerDist.Pool = proposerDist.Pool.Plus(remaining)
+	proposerDist.ValCommission = proposerDist.ValCommission.Plus(commission)


hmm - maybe CommissionRewards instead? I'd like to imply that it contains coins and is not a commission rate

CommissionPool? Or is that confusing since only one validator can withdraw. I think CommissionRewards is fine as well.

rigelrozanski · 2018-10-26T08:28:17Z

Close in favour of #2597 which has merged this branch?

cwgoes

Minor comments.

cwgoes · 2018-10-26T11:18:17Z

x/distribution/keeper/allocation.go

@@ -27,8 +27,8 @@ func (k Keeper) AllocateTokens(ctx sdk.Context, percentVotes sdk.Dec, proposer s
 	// apply commission
 	commission := proposerReward.MulDec(proposerValidator.GetCommission())
 	remaining := proposerReward.Minus(commission)
-	proposerDist.PoolCommission = proposerDist.PoolCommission.Plus(commission)
-	proposerDist.Pool = proposerDist.Pool.Plus(remaining)
+	proposerDist.ValCommission = proposerDist.ValCommission.Plus(commission)


CommissionPool? Or is that confusing since only one validator can withdraw. I think CommissionRewards is fine as well.

cwgoes · 2018-10-26T11:19:21Z

x/distribution/keeper/hooks.go

 			panic(err)
 		}
 	}
 }

+// XXX Consider removing this after debugging.


I think we can leave it if you like, it's a very cheap sanity check

cwgoes · 2018-10-26T11:19:46Z

x/distribution/keeper/hooks.go

 }
-func (h Hooks) OnValidatorBonded(ctx sdk.Context, _ sdk.ConsAddress, addr sdk.ValAddress) {
-	h.k.onValidatorModified(ctx, addr)
+func (h Hooks) OnValidatorPowerDidChange(ctx sdk.Context, _ sdk.ConsAddress, valAddr sdk.ValAddress) {


This name is confusing to me - this gets called after the validator's power has changed I guess?

x/stake/keeper/slash.go

cwgoes

Tested ACK

cwgoes · 2018-10-26T11:30:54Z

Close in favour of #2597 which has merged this branch?

Ah sorry, somehow I missed this comment. I merged this into develop and develop into your branch.

@rigelrozanski

* Back to 50 initially bonded * Add query deposits cmds * Update test * Update PENDING.md * Back to on-operation=false, update PENDING.md * Remove unnecessary change, clarify amt in sim_test.go * Cleanup, additional item in PENDING.md * Update PENDING.md Co-Authored-By: alessio <quadrispro@ubuntu.com> * Update client/keys/utils.go Co-Authored-By: alessio <quadrispro@ubuntu.com> * update depositer addr * Address @rigelrozanski comments * Make linter happy * Update PENDING.md * Remove shorthand * Make "multi" const * Store last total power as sdk.Int, not sdk.Dec * Merge PR cosmos#2553: Renamed msg.Name() and msg.Type() to msg.Type() and msg.Route() * Fix stupid mistake * s/number/weight/ * LastValidatorPower is also an Int * Merge PR cosmos#2462: Add swagger-ui for gov, stake and slashing * 'make format' * Move PENDING to CHANGELOG * Linkify changelog * Fix db write perm * Additional cleanup * Remove logs from prior debugging * Merge PR cosmos#2599 from cosmos/jae/dist_refactor * Rename Pool -> DelRewards; PoolCommission -> ValCommision * FeePool.Pool -> FeePool.ValPool * WithdrawalHeight->DelPoolWithdrawalHeight * OnValidatorBeginUnbonding * Caught the bug's tail * Update vi.FeePoolWithdrawalHeight upon bonding * Fix staking slashUnbondingDelegation bug; fixes simulator failure cosmos#9 * Merge PR cosmos#2597: Add distribution accum invariants * PENDING.md => CHANGELOG.md * Manually linkify * Manually fix some links * Docs fixes in progress while running through the release process * More docs fixes * '--voter' is no longer required * Rectify validator setup documentation * Merge PR cosmos#2596: Cmds for validator unbondings and redelegations * Make simulation use a transition matrix for block size This enables simulating periods of high load, and periods of low to no load. (low load because future ops will still terminate in that time frame) * address bez's comments * fix flags in docs, closes cosmos#2530 * Merge PR cosmos#2616: Block redelegations to the same validator * Merge PR cosmos#2623: Speedup simulator by switching to goleveldb Due to requiring app.Commit() at the moment, golevel db is significantly faster than a memdb * fix block offsets in printing simulation block number * Merge PR cosmos#2644: Simulation: Print last block when there is an error There was an off by one error in the log printing function previously * Merge PR cosmos#2642: Add todo diagrams * Various sign command improvements - Exit with error if the user is attempting to sign with a key whose address is not among those who are expected to sign the transaction. - Add --print-signature-only to output only the generated signature. * Check sanity of signatures and report errors when run with --print-sigs * Improve errors reporting * Improve online docs * Refresh PENDING.md * Find better name for --print-signature-only * Fix integration tests * Validate --name * Fix integration tests * s/--print-sigs/--validate-signatures/ * s/--sig-only/--signature-only/ * Docs updated * Update PENDING.md * Rename append, it's go builtin * Set success = false when it fails * Apply suggestions from bez * Nest switches * Fix rebase * Document what --validate-signatures does * perform minor doc and function cleanup * move typedef * Merge PR cosmos#2614: Configurable Bech32 prefix for SDK users * Merge PR cosmos#2643: AppendTag function usage error. append elements do not work * simulation: Make validator choice use validator set This also had to change the default seed, since with the previous one it actually got into a state where there were no validators left bonded, lol. This also changes Unbond msgs from failing with almost 100% probability to now only failing with 33% probability. Thus more of the state machine is getting tested! * Update changelog * Merge PR cosmos#2657: Fix config.js * Merge PR cosmos#2589: Update Vesting Spec * Merge PR cosmos#2656: Revert read-only leveldb database * Revert read-only leveldb database Waiting on a fix for syndtr/goleveldb#240. * Update client/keys/utils.go * Include DNS alt name in certificate Closes: cosmos#2664 * Gaialite signal handling is broken, repair it * Merge PR cosmos#2665: simulation: Remove header from Invariant This got introduced recently, but wasn't actually needed, hence the reversion * Merge PR cosmos#2653: Add benchmark for get and set account * Fix test * Refactor TrapSignal * Fix lint * enforcing @jaekwon mergemaster * added querier redelegation * added validatorDelegations querier endpoint * LCD and CLI * cli fixes * removed redelegation stuff * address other comments * rebased * addressed comments * Make the simulator create the new comission rate sensibly * Update to TM v0.26.0 - Part I (cosmos#2679) * Update to TM v0.26.0 * Bez/tm0.26 update pt 2 redux (cosmos#2684) * Update to TM v0.26.0 * Update TODOs * Proof and verification updates * Fix linting * Fix key path creation * Temporarily fix tendermint revision to make tests pass * Fix merge conflict bug; Update PENDING * New genesis workflow (cosmos#2602) New genesis workflow: * `gaiad init` is now used to generate an empty `genesis.json`. * Genesis accounts need to be populated manually before running `gaiad collect-gentxs`. * This should support starfish too, see cosmos#2615 for more info. * Closes: cosmos#2596 cosmos#2615 * Validate validator address and address against respective account ex ante * Fix local testnet failures * New genesis tests * Run make format * Add --pubkey flag * gaiad collect-gentxs takes no args * Simulation improvements (logging fix, random genesis parameters) (cosmos#2617) * Print out initial update on every block * Randomize simulation parameters * Randomize initial liveness weightings * Randomize genesis parameters * fixed power store invariant * IterateValidatorsBonded -> IterateBondedValidatorsByPower * WriteValidators uses IterateLastValidators rather than IterateBondedValidatorsByPower * fixed democoin interface Closes cosmos#2556 Closes cosmos#2396 Via cosmos#2671: closes cosmos#2669 closes cosmos#2670 closes cosmos#2620 Offshoot issues: cosmos#2618 cosmos#2619 cosmos#2620 cosmos#2661 * Fix simulation bugs; Incorprates cosmos#2676 from Sunny (cosmos#2677) * Fix simulation bugs; Incorprates cosmos#2676 from Sunny * Address review feedback; Update PENDING * 'make format' * Revert "enforcing @jaekwon mergemaster" This reverts commit 15c2093. * Update x/stake/client/rest/query.go Co-Authored-By: sunnya97 <sunnya97@gmail.com> * addressed fede's comment * Switch gov proposal-queues to use iterators (cosmos#2638) * switched gov proposals queue to use iterators * update gov spec * update proposal.Equal * Amino api change * switched proposalID to uint64 * renamed Gov Procedures to Params * s/ActiveProposalQueueProposalKey/KeyActiveProposalQueueProposal/g * numLatestProposals -> Limit * fixed staking invariant breakage because of gov deposits * Send deposits to DepositedCoinsAccAddr or BurnedDepositCoinsAccAddr * Add general merkle absence proof (also for empty substores) (cosmos#2685) * Fix coins.IsLT() impl (cosmos#2686) * Fix coins.IsLT() impl * Fix coin.IsLT() impl * Coins.IsLT -> Coins.IsAllLT etc * Update testnet to use canonical genesis time (cosmos#2692) * Update testnet to use canonical genesis time * Fix linting in genesis test * Do not allow nil values to be set in CacheKVStore (cosmos#2708) * Do not allow nil values to be set in CacheKVStore * Makefile OS compatibility update * Merge PR cosmos#2714: Add commission data to MsgCreateValidator signature bytes * PENDING => CHANGELOG * Linkify changelog * Cleanup bank keeper * whitespacing * rand utile ... * moving stuff around a bit, trying to get rid of types * reorganize more * rename ambig naming of queueOperations * minimizing indentation * fix some duplicate to get passing * Address style comments * Reorganize CLI command structure. Fixes cosmos#2575 * Fix missing flags issue * Address linting issues * Fix gobash CLI testing * Fix typo * Cross-compiling get_tools Makefile added * operations functions * assertAllInvarients changes, Operation reorg * mock tendermint * util cleanup * event stats object, more general cleanup * compiling * pending * Removed comment from Makefile as per bez's request * val comments * Address PR comments * Update cmd/gaia/cmd/gaiacli/main.go Co-Authored-By: jackzampolin <jack.zampolin@gmail.com> * PENDING * Fix state export/import, add to CI (cosmos#2690) * Update slashing import/export * More slashing.WriteGenesis * Add test import/export to CI * Store equality comparison. * Fix validator bond intra-tx counter * Set timeslices for unbonding validators * WriteGenesis => ExportGenesis * Delete validators from unbonding queue when re-bonded * Hook for validator deletion, fix staking genesis tests * Merge 0.26.0 back to develop (cosmos#2718) * PENDING => CHANGELOG * Linkify changelog * Merge PR cosmos#2716: Temporarily disable gaia lite insecure mode * TODO: need to update CHANGELOG w/ import-export PR cosmos#2690 * Update CHANGELOG/PENDING for straggling PR cosmos#2690 * Add small utility to add account to genesis.json after gaiad init * Update CHANGELOG.md * s/WriteGenesisFile/ExportGenesisFile/ * Update PENDING.md * Add --chain-id to testnet command * Address remaining comments from cosmos#2690 * Update PENDING.md * add back in PeriodicInvariant * Linter fix * Fix TimeoutCommit (cosmos#2743) * Fix TimeoutCommit to 5 seconds instead of whatever it was before which was too short. * Gaia-9000: Update to TM 0.26.1-rc2 (cosmos#2753) * Update to tm 0.26.1-rc2 to fix prometheus issue and node disconnect issue. * Gaia-9000: Update to TM 0.26.1-rc3 -- pex SeedMode fix * fix typo I think it might be a spelling mistake * Slight distribution spec cleanup * More cleanup * use defer * Use correct Bech32 prefix for show-address command (cosmos#2746) * Use consensus address bech32 prefix * Update show-address CLI description * Generate random moniker when missing * Update moniker prefix * Require moniker instead of generating a random one * update to tendermint v0.26.1 * Fix test coverage * Correctly set return code * Fix date to be cross platform * Merge PR cosmos#2752: Don't hardcode bondable denom * R4R: Fix unbonding command flow (cosmos#2727) * Fix required flag * Fix redelegation command * Add pending entry * update swagger.yaml * use newQuery...Params * Link to issue * Fix DiffKVStore * Address PR review * Working on stake import/export * Only apply validator set updates on initial genesis * Clarify comment * Fix failing test * add back in CLI command after rebase * Fix CLI tests * update to amino 0.14.1 * pending * R4R: Query Gov Params (cosmos#2576) * gov query params * Merge PR cosmos#2744: Fix Makefile targets dependencies * Fix Makefile targets dependencies * Remove unnecessary build deps from install targets * Create a rule for each tool * Don't dep test_lint on tools * Update docs/spec/distribution/overview.md Co-Authored-By: alexanderbez <alexanderbez@users.noreply.github.com> * Update docs/spec/distribution/overview.md Co-Authored-By: alexanderbez <alexanderbez@users.noreply.github.com> * Update docs/spec/distribution/overview.md Co-Authored-By: alexanderbez <alexanderbez@users.noreply.github.com> * Update overview.md * Documentation Structure Change and Cleanup (cosmos#2808) * Update docs/sdk/clients.md * organize ADR directory like tendermint * docs: move spec-proposals into spec/ * remove lotion, moved to website repo * move getting-started to cosmos-hub, and voyager to website * docs: move lite/ into clients/lite/ * move introduction/ content to website repo * move resources/ content to website repo * mv sdk/clients.md to clients/clients.md * mv validators to cosmos-hub/validators * move deprecated sdk/ content to _attic * sdk/modules.md is duplicate with modules/README.md * consolidate remianing sdk/ files into a single sdk.md * move examples/ to docs/examples/ * mv docs/cosmos-hub to docs/gaia * Add keys/accounts section to localnet docs * Bring back banner (cosmos#2814) * Build docs in CircleCI (cosmos#2810) * error checking the API call * added docs build trigger to circleci job * Update contributing.md with new merge policy (cosmos#2789) * Update contribuiting.md with new merge policy * deleted obsolete file (cosmos#2817)

jaekwon added 5 commits October 25, 2018 16:34

Rename Pool -> DelRewards; PoolCommission -> ValCommision

e23054f

FeePool.Pool -> FeePool.ValPool

93d5437

WithdrawalHeight->DelPoolWithdrawalHeight

33521b0

OnValidatorBeginUnbonding

1143c93

Caught the bug's tail

c17f425

jaekwon requested review from cwgoes, ebuchman and rigelrozanski as code owners October 26, 2018 00:52

rigelrozanski reviewed Oct 26, 2018

View reviewed changes

Update vi.FeePoolWithdrawalHeight upon bonding

b97a076

rigelrozanski mentioned this pull request Oct 26, 2018

WIP: distribution refactor #2578

Closed

5 tasks

Fix staking slashUnbondingDelegation bug; fixes simulator failure #9

99efde2

cwgoes reviewed Oct 26, 2018

View reviewed changes

cwgoes approved these changes Oct 26, 2018

View reviewed changes

cwgoes merged commit 0f1fb17 into develop Oct 26, 2018

cwgoes deleted the jae/dist_refactor branch October 26, 2018 11:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP Exploring x/distribution cleanup #2599

WIP Exploring x/distribution cleanup #2599

jaekwon commented Oct 26, 2018

codecov bot commented Oct 26, 2018 •

edited

Loading

rigelrozanski Oct 26, 2018

rigelrozanski Oct 26, 2018

cwgoes Oct 26, 2018

rigelrozanski commented Oct 26, 2018

cwgoes left a comment

cwgoes Oct 26, 2018

cwgoes Oct 26, 2018

cwgoes Oct 26, 2018

cwgoes left a comment

cwgoes commented Oct 26, 2018

WIP Exploring x/distribution cleanup #2599

WIP Exploring x/distribution cleanup #2599

Conversation

jaekwon commented Oct 26, 2018

codecov bot commented Oct 26, 2018 • edited Loading

Codecov Report

rigelrozanski Oct 26, 2018

Choose a reason for hiding this comment

rigelrozanski Oct 26, 2018

Choose a reason for hiding this comment

cwgoes Oct 26, 2018

Choose a reason for hiding this comment

rigelrozanski commented Oct 26, 2018

cwgoes left a comment

Choose a reason for hiding this comment

cwgoes Oct 26, 2018

Choose a reason for hiding this comment

cwgoes Oct 26, 2018

Choose a reason for hiding this comment

cwgoes Oct 26, 2018

Choose a reason for hiding this comment

cwgoes left a comment

Choose a reason for hiding this comment

cwgoes commented Oct 26, 2018

codecov bot commented Oct 26, 2018 •

edited

Loading