DNM: Adhoc invariant checks {LATEST debugging branch} #3759

cwgoes · 2019-02-27T16:22:54Z

More debugging, DNM.

cwgoes · 2019-02-27T16:26:06Z

Now fails almost immediately on most seeds.

cwgoes · 2019-02-27T17:09:08Z

... now this looks like a rounding issue (could be separate from the other failure though)

cwgoes · 2019-02-27T17:54:32Z

TestMsgSetWithdrawAddress: ok false, msg {"type":"cosmos-sdk/MsgModifyWithdrawAddress","value":{"delegator_address":"cosmos133425u4tnsm57lssa426xq8vml3zqxcwen7h0j","withdraw_address":"cosmos1da3kj2a85dvlrq0nszluahy9l62xarmk5jvls3"}}
TestMsgCreateValidator: ok false, msg {"type":"cosmos-sdk/MsgCreateValidator","value":{"commission":{"max_change_rate":"0.154054377639072113","max_rate":"0.343000000000000000","rate":"0.070896044266805983"},"delegator_address":"cosmos133425u4tnsm57lssa426xq8vml3zqxcwen7h0j","description":{"details":"","identity":"","moniker":"NOgIgWOqyf","website":""},"min_self_delegation":"1","pubkey":"cosmosvalconspub1zcjduepq34dr35dtfl3e82grpvx4rf82zus6nkdvs9x8pyajwvvjn755vc9qlt9wy8","validator_address":"cosmosvaloper133425u4tnsm57lssa426xq8vml3zqxcwu82zrp","value":{"amount":"247864064043","denom":"stake"}}}
cosmos1qmjs80th0x20zpd9hzyyuaxx0395e5qjjq9w8c is sending 538497393970 stake to cosmos133425u4tnsm57lssa426xq8vml3zqxcwen7h0j
TestMsgBeginRedelegate: {"type":"cosmos-sdk/MsgBeginRedelegate","value":{"delegator_address":"cosmos133425u4tnsm57lssa426xq8vml3zqxcwen7h0j","shares_amount":"986864217889.000000000000000000","validator_dst_address":"cosmosvaloper1gwwg6eve0wcwnwj8axrfwm4lqnll74tz43my0d","validator_src_address":"cosmosvaloper1gwwg6eve0wcwnwj8axrfwm4lqnll74tz43my0d"}}
TestMsgWithdrawDelegatorReward: ok false, msg {"type":"cosmos-sdk/MsgWithdrawDelegationReward","value":{"delegator_address":"cosmos133425u4tnsm57lssa426xq8vml3zqxcwen7h0j","validator_address":"cosmosvaloper1a890au2q9rgggqajenjqnln7qy3226ysc9azpe"}}
TestMsgDelegate: ok true, msg {"type":"cosmos-sdk/MsgDelegate","value":{"delegator_address":"cosmos133425u4tnsm57lssa426xq8vml3zqxcwen7h0j","validator_address":"cosmosvaloper14z56f9hpj5cw0qv6arzgnjsvz9k4musmppc7mq","value":{"amount":"240658330020","denom":"stake"}}}
cosmos1qu379fd7lzvl9pfwclw2984n99dfqdgxhs43dh is sending 19408964530 stake to cosmos133425u4tnsm57lssa426xq8vml3zqxcwen7h0j
cosmos133425u4tnsm57lssa426xq8vml3zqxcwen7h0j is sending 557439758110 stake to cosmos1yzyr8s3v3g26jqzxtx73sr0fe698rzkez6l65e
TestMsgSetWithdrawAddress: ok false, msg {"type":"cosmos-sdk/MsgModifyWithdrawAddress","value":{"delegator_address":"cosmos133425u4tnsm57lssa426xq8vml3zqxcwen7h0j","withdraw_address":"cosmos18xeg0kakety7835527tpwzu4zlchufcedc20tv"}}
cosmos133425u4tnsm57lssa426xq8vml3zqxcwen7h0j is sending 42150008107 stake to cosmos1da3kj2a85dvlrq0nszluahy9l62xarmk5jvls3
TestMsgBeginRedelegate: {"type":"cosmos-sdk/MsgBeginRedelegate","value":{"delegator_address":"cosmos133425u4tnsm57lssa426xq8vml3zqxcwen7h0j","shares_amount":"19732291918.000000000000000000","validator_dst_address":"cosmosvaloper1yzyr8s3v3g26jqzxtx73sr0fe698rzke8wt0c2","validator_src_address":"cosmosvaloper1l67uvpuauv6wd90rvuln8ywp3trwfcc6csx0hn"}}
cosmos14z56f9hpj5cw0qv6arzgnjsvz9k4musmy4vthn is sending 394086373820 stake to cosmos133425u4tnsm57lssa426xq8vml3zqxcwen7h0j
cosmos133425u4tnsm57lssa426xq8vml3zqxcwen7h0j is sending 236017005169 stake to cosmos1qmjs80th0x20zpd9hzyyuaxx0395e5qjjq9w8c
cosmos1yzyr8s3v3g26jqzxtx73sr0fe698rzkez6l65e is sending 60619380460 stake to cosmos133425u4tnsm57lssa426xq8vml3zqxcwen7h0j
cosmos1hqp3gsg4362g8693s5tytx9p3tszacxcaf3cwn is sending 91645024641 stake to cosmos133425u4tnsm57lssa426xq8vml3zqxcwen7h0j
TestMsgDelegate: ok true, msg {"type":"cosmos-sdk/MsgDelegate","value":{"delegator_address":"cosmos133425u4tnsm57lssa426xq8vml3zqxcwen7h0j","validator_address":"cosmosvaloper133425u4tnsm57lssa426xq8vml3zqxcwu82zrp","value":{"amount":"376303244434","denom":"stake"}}}
TestMsgCreateValidator: ok false, msg {"type":"cosmos-sdk/MsgCreateValidator","value":{"commission":{"max_change_rate":"0.462294971930853696","max_rate":"0.469000000000000000","rate":"0.000000000000000000"},"delegator_address":"cosmos133425u4tnsm57lssa426xq8vml3zqxcwen7h0j","description":{"details":"","identity":"","moniker":"DVLrGvyZHL","website":""},"min_self_delegation":"1","pubkey":"cosmosvalconspub1zcjduepq34dr35dtfl3e82grpvx4rf82zus6nkdvs9x8pyajwvvjn755vc9qlt9wy8","validator_address":"cosmosvaloper133425u4tnsm57lssa426xq8vml3zqxcwu82zrp","value":{"amount":"17066319537","denom":"stake"}}}
TestMsgWithdrawDelegatorReward: ok false, msg {"type":"cosmos-sdk/MsgWithdrawDelegationReward","value":{"delegator_address":"cosmos133425u4tnsm57lssa426xq8vml3zqxcwen7h0j","validator_address":"cosmosvaloper1ddk40t847eqyyuxedled0kz3alvk4jvvjc34yd"}}
TestMsgBeginRedelegate: {"type":"cosmos-sdk/MsgBeginRedelegate","value":{"delegator_address":"cosmos133425u4tnsm57lssa426xq8vml3zqxcwen7h0j","shares_amount":"20404654909.000000000000000000","validator_dst_address":"cosmosvaloper1yzyr8s3v3g26jqzxtx73sr0fe698rzke8wt0c2","validator_src_address":"cosmosvaloper1a890au2q9rgggqajenjqnln7qy3226ysc9azpe"}}
TestMsgUndelegate: ok true, msg {"type":"cosmos-sdk/MsgUndelegate","value":{"delegator_address":"cosmos133425u4tnsm57lssa426xq8vml3zqxcwen7h0j","shares_amount":"935489750407.363133209504593180","validator_address":"cosmosvaloper133425u4tnsm57lssa426xq8vml3zqxcwu82zrp"}}
TestMsgSetWithdrawAddress: ok false, msg {"type":"cosmos-sdk/MsgModifyWithdrawAddress","value":{"delegator_address":"cosmos133425u4tnsm57lssa426xq8vml3zqxcwen7h0j","withdraw_address":"cosmos14z56f9hpj5cw0qv6arzgnjsvz9k4musmy4vthn"}}
TestMsgDelegate: ok true, msg {"type":"cosmos-sdk/MsgDelegate","value":{"delegator_address":"cosmos133425u4tnsm57lssa426xq8vml3zqxcwen7h0j","validator_address":"cosmosvaloper14z56f9hpj5cw0qv6arzgnjsvz9k4musmppc7mq","value":{"amount":"177549347033","denom":"stake"}}}
TestMsgUndelegate: ok false, msg {"type":"cosmos-sdk/MsgUndelegate","value":{"delegator_address":"cosmos133425u4tnsm57lssa426xq8vml3zqxcwen7h0j","shares_amount":"28069599174.718897307821102581","validator_address":"cosmosvaloper133425u4tnsm57lssa426xq8vml3zqxcwu82zrp"}}
TestMsgWithdrawDelegatorReward: ok false, msg {"type":"cosmos-sdk/MsgWithdrawDelegationReward","value":{"delegator_address":"cosmos133425u4tnsm57lssa426xq8vml3zqxcwen7h0j","validator_address":"cosmosvaloper1a890au2q9rgggqajenjqnln7qy3226ysc9azpe"}}
TestMsgBeginRedelegate: {"type":"cosmos-sdk/MsgBeginRedelegate","value":{"delegator_address":"cosmos133425u4tnsm57lssa426xq8vml3zqxcwen7h0j","shares_amount":"6775168564.000000000000000000","validator_dst_address":"cosmosvaloper17mfwlupxfc0pe0ccf80yhmlzchduvla5yzrdg7","validator_src_address":"cosmosvaloper1dnyxfepmapaz0k9l5gu09sr0dpvr3phwqmvry3"}}
TestMsgUndelegate: ok false, msg {"type":"cosmos-sdk/MsgUndelegate","value":{"delegator_address":"cosmos133425u4tnsm57lssa426xq8vml3zqxcwen7h0j","shares_amount":"32570739332.692744323091864934","validator_address":"cosmosvaloper133425u4tnsm57lssa426xq8vml3zqxcwu82zrp"}}
TestMsgCreateValidator: ok false, msg {"type":"cosmos-sdk/MsgCreateValidator","value":{"commission":{"max_change_rate":"0.037000000000000000","max_rate":"0.037000000000000000","rate":"0.019497370003603742"},"delegator_address":"cosmos133425u4tnsm57lssa426xq8vml3zqxcwen7h0j","description":{"details":"","identity":"","moniker":"HUrykdQExj","website":""},"min_self_delegation":"1","pubkey":"cosmosvalconspub1zcjduepq34dr35dtfl3e82grpvx4rf82zus6nkdvs9x8pyajwvvjn755vc9qlt9wy8","validator_address":"cosmosvaloper133425u4tnsm57lssa426xq8vml3zqxcwu82zrp","value":{"amount":"11497084315","denom":"stake"}}}
TestMsgCreateValidator: ok false, msg {"type":"cosmos-sdk/MsgCreateValidator","value":{"commission":{"max_change_rate":"0.208956747399952253","max_rate":"0.360000000000000000","rate":"0.360000000000000000"},"delegator_address":"cosmos133425u4tnsm57lssa426xq8vml3zqxcwen7h0j","description":{"details":"","identity":"","moniker":"hAuKvMPPom","website":""},"min_self_delegation":"1","pubkey":"cosmosvalconspub1zcjduepq34dr35dtfl3e82grpvx4rf82zus6nkdvs9x8pyajwvvjn755vc9qlt9wy8","validator_address":"cosmosvaloper133425u4tnsm57lssa426xq8vml3zqxcwu82zrp","value":{"amount":"8070780119","denom":"stake"}}}
cosmos1ddk40t847eqyyuxedled0kz3alvk4jvvhv9qg7 is sending 289640039 stake to cosmos133425u4tnsm57lssa426xq8vml3zqxcwen7h0j

cwgoes · 2019-02-27T17:57:38Z

Filtered:

TestMsgDelegate: ok true, msg {"type":"cosmos-sdk/MsgDelegate","value":{"delegator_address":"cosmos133425u4tnsm57lssa426xq8vml3zqxcwen7h0j","validator_address":"cosmosvaloper14z56f9hpj5cw0qv6arzgnjsvz9k4musmppc7mq","value":{"amount":"240658330020","denom":"stake"}}}
TestMsgDelegate: ok true, msg {"type":"cosmos-sdk/MsgDelegate","value":{"delegator_address":"cosmos133425u4tnsm57lssa426xq8vml3zqxcwen7h0j","validator_address":"cosmosvaloper133425u4tnsm57lssa426xq8vml3zqxcwu82zrp","value":{"amount":"376303244434","denom":"stake"}}}
TestMsgUndelegate: ok true, msg {"type":"cosmos-sdk/MsgUndelegate","value":{"delegator_address":"cosmos133425u4tnsm57lssa426xq8vml3zqxcwen7h0j","shares_amount":"935489750407.363133209504593180","validator_address":"cosmosvaloper133425u4tnsm57lssa426xq8vml3zqxcwu82zrp"}}
TestMsgDelegate: ok true, msg {"type":"cosmos-sdk/MsgDelegate","value":{"delegator_address":"cosmos133425u4tnsm57lssa426xq8vml3zqxcwen7h0j","validator_address":"cosmosvaloper14z56f9hpj5cw0qv6arzgnjsvz9k4musmppc7mq","value":{"amount":"177549347033","denom":"stake"}}}

cwgoes · 2019-02-27T22:03:19Z

(todo: rebase once #3750 is merged to develop)

x/distribution/keeper/hooks.go

cwgoes · 2019-02-27T23:39:10Z

.... now fails earlier on a hook, thanks @rigelrozanski

rigelrozanski · 2019-02-28T02:33:14Z

@cwgoes I’m taking a break from this debugging for a while… my head is spinning, the panicing new test is flawed though it doesn’t take into account the community pool changes (but I can’t seem to set it up correctly) see some of the changes I made in my debugging branch https://github.com/cosmos/cosmos-sdk/blob/30658ef8893ab2969c83f3d0be800812b1f55906/x/distribution/keeper/allocation.go#L25……… I suspect that this check will not catch the bug and once we rectify this test the bug will still persist elsewhere

…ection

cwgoes · 2019-02-28T15:58:41Z

edit: Ignore this

Minimal reproduction:

go test ./x/distribution/keeper/... -v -run TestAllocateTokensTruncation

Results in:

=== RUN   TestAllocateTokensTruncation
INIT cosmos16avry6hvj4hnmtlvqzltmdml06rg2k64ypl4a4 with shares 110.000000000000000000000000000000000000000000000000000000000000, tokens 110.000000000000000000000000000000000000000000000000000000000000
INIT cosmos1lfyrjphpp05xs56cxnn6d7grqt9p2p8zttm08a with shares 100.000000000000000000000000000000000000000000000000000000000000, tokens 100.000000000000000000000000000000000000000000000000000000000000
withdraw for delegator: cosmos1lfyrjphpp05xs56cxnn6d7grqt9p2p8zttm08a
startingPeriod 1, endingPeriod 2, difference: , stake 100.000000000000000000000000000000000000000000000000000000000000, rewards: 
INIT cosmos1lfyrjphpp05xs56cxnn6d7grqt9p2p8zttm08a with shares 100.000000000000000000000000000000000000000000000000000000000000, tokens 100.000000000000000000000000000000000000000000000000000000000000
allocating 31709792.000000000000000000000000000000000000000000000000000000000000stake tokens to validator cosmosvaloper1lfyrjphpp05xs56cxnn6d7grqt9p2p8zwl06tw, prior withdrawable: 
withdraw for delegator: cosmos1lfyrjphpp05xs56cxnn6d7grqt9p2p8zttm08a
startingPeriod 1, endingPeriod 2, difference: 285388.128000000000000000000000000000000000000000000000000000000000stake, stake 100.000000000000000000000000000000000000000000000000000000000000, rewards: 28538812.800000000000000000000000000000000000000000000000000000000000stake
INIT cosmos1lfyrjphpp05xs56cxnn6d7grqt9p2p8zttm08a with shares 100.000000000000000000000000000000000000000000000000000000000000, tokens 100.000000000000000000000000000000000000000000000000000000000000
withdraw for delegator: cosmos16avry6hvj4hnmtlvqzltmdml06rg2k64ypl4a4
startingPeriod 1, endingPeriod 2, difference: , stake 110.000000000000000000000000000000000000000000000000000000000000, rewards: 
INIT cosmos16avry6hvj4hnmtlvqzltmdml06rg2k64ypl4a4 with shares 110.000000000000000000000000000000000000000000000000000000000000, tokens 110.000000000000000000000000000000000000000000000000000000000000
allocating 213785371.870967741935483870967741935483870967741935483870967528150112stake tokens to validator cosmosvaloper16avry6hvj4hnmtlvqzltmdml06rg2k64p4tq3x, prior withdrawable: 
withdraw for delegator: cosmos16avry6hvj4hnmtlvqzltmdml06rg2k64ypl4a4
startingPeriod 1, endingPeriod 2, difference: 1749153.042580645161290322580645161290322580645161290322580643412137stake, stake 110.000000000000000000000000000000000000000000000000000000000000, rewards: 192406834.683870967741935483870967741935483870967741935483870775335070stake
INIT cosmos16avry6hvj4hnmtlvqzltmdml06rg2k64ypl4a4 with shares 110.000000000000000000000000000000000000000000000000000000000000, tokens 110.000000000000000000000000000000000000000000000000000000000000
withdraw for delegator: cosmos1lfyrjphpp05xs56cxnn6d7grqt9p2p8zttm08a
startingPeriod 1, endingPeriod 2, difference: 285388.128000000000000000000000000000000000000000000000000000000000stake, stake 100.000000000000000000000000000000000000000000000000000000000000, rewards: 28538812.800000000000000000000000000000000000000000000000000000000000stake
INIT cosmos1lfyrjphpp05xs56cxnn6d7grqt9p2p8zttm08a with shares 100.000000000000000000000000000000000000000000000000000000000000, tokens 100.000000000000000000000000000000000000000000000000000000000000
allocating 194350338.064516129032258064516129032258064516129032258064515934681920stake tokens to validator cosmosvaloper1lfyrjphpp05xs56cxnn6d7grqt9p2p8zwl06tw, prior withdrawable: 31709791.800000000000000000000000000000000000000000000000000000000000stake
withdraw for delegator: cosmos1lfyrjphpp05xs56cxnn6d7grqt9p2p8zttm08a
startingPeriod 1, endingPeriod 2, difference: 2034541.170580645161290322580645161290322580645161290322580643412137stake, stake 100.000000000000000000000000000000000000000000000000000000000000, rewards: 203454117.058064516129032258064516129032258064516129032258064341213700stake
INIT cosmos1lfyrjphpp05xs56cxnn6d7grqt9p2p8zttm08a with shares 100.000000000000000000000000000000000000000000000000000000000000, tokens 100.000000000000000000000000000000000000000000000000000000000000
--- FAIL: TestAllocateTokensTruncation (0.00s)
panic: greater withdraw allowed than allocated: validator cosmosvaloper1lfyrjphpp05xs56cxnn6d7grqt9p2p8zwl06tw, allowed: 194350338.258064516129032258064516129032258064516129032258064341213700stake, allocated 194350338.064516129032258064516129032258064516129032258064515934681920stake
 [recovered]
	panic: greater withdraw allowed than allocated: validator cosmosvaloper1lfyrjphpp05xs56cxnn6d7grqt9p2p8zwl06tw, allowed: 194350338.258064516129032258064516129032258064516129032258064341213700stake, allocated 194350338.064516129032258064516129032258064516129032258064515934681920stake


goroutine 25 [running]:
testing.tRunner.func1(0xc0001fab00)
	/usr/lib/go/src/testing/testing.go:792 +0x387
panic(0xb38fa0, 0xc0000e09e0)
	/usr/lib/go/src/runtime/panic.go:513 +0x1b9
github.com/cosmos/cosmos-sdk/x/distribution/keeper.Keeper.AllocateTokensToValidator(0xe47100, 0xc000010b30, 0xc000122d90, 0xc000122d90, 0xe47100, 0xc000010b80, 0xe47140, 0xc000010b90, 0xc000b34720, 0x5, ...)
	/home/cwgoes/working/go/src/github.com/cosmos/cosmos-sdk/x/distribution/keeper/allocation.go:125 +0xce7
github.com/cosmos/cosmos-sdk/x/distribution/keeper.Keeper.AllocateTokens(0xe47100, 0xc000010b30, 0xc000122d90, 0xc000122d90, 0xe47100, 0xc000010b80, 0xe47140, 0xc000010b90, 0xc000b34720, 0x5, ...)
	/home/cwgoes/working/go/src/github.com/cosmos/cosmos-sdk/x/distribution/keeper/allocation.go:85 +0xa70
github.com/cosmos/cosmos-sdk/x/distribution/keeper.TestAllocateTokensTruncation(0xc0001fab00)
	/home/cwgoes/working/go/src/github.com/cosmos/cosmos-sdk/x/distribution/keeper/allocation_test.go:156 +0x1163
testing.tRunner(0xc0001fab00, 0xda9f50)
	/usr/lib/go/src/testing/testing.go:827 +0xbf
created by testing.(*T).Run
	/usr/lib/go/src/testing/testing.go:878 +0x35c
FAIL	github.com/cosmos/cosmos-sdk/x/distribution/keeper	0.039s

cwgoes · 2019-02-28T16:38:46Z

... ok, increasing decimal precision fixes the sim

rigelrozanski · 2019-02-28T17:14:07Z

types/decimal.go


 	// bytes required to represent the above precision
 	// Ceiling[Log2[999 999 999 999 999 999]]
-	DecimalPrecisionBits = 60
+	DecimalPrecisionBits = 139


should be 200

(this was updated)

cwgoes · 2019-02-28T17:17:46Z

Now a different invariant breaks:

go test ./cmd/gaia/app -run TestFullGaiaSimulation -SimulationEnabled=true -SimulationNumBlocks=50 -SimulationVerbose=true -SimulationCommit=true -SimulationSeed=44 -v -timeout 24h

results in

^MSimulating... block 10/50, operation 0/186. ^MSimulating... block 10/50, operation 0/88. ^MSimulating... block 10/50, operation 50/88. Panic with err
 invariant broken: loose token invariance:
  pool.NotBondedTokens: 154156533082310
  sum of account tokens: 154152954180948.573861757546052947916403292097357630096408412959470572950135
goroutine 6 [running]:
runtime/debug.Stack(0xc004c26750, 0x2, 0x2) 
  /usr/lib/go/src/runtime/debug/stack.go:24 +0xa7
github.com/cosmos/cosmos-sdk/x/mock/simulation.SimulateFromSeed.func2(0xc0017c2020, 0xc0004ca960, 0xc0000e4fd0)
  /home/cwgoes/working/go/src/github.com/cosmos/cosmos-sdk/x/mock/simulation/simulate.go:131 +0xb6
panic(0xcb1f60, 0xc002e79ca0)
  /usr/lib/go/src/runtime/panic.go:513 +0x1b9
github.com/cosmos/cosmos-sdk/cmd/gaia/app.(*GaiaApp).assertRuntimeInvariantsOnContext(0xc000184d80, 0xfe9600, 0xc0013ef950, 0xc0009da3c0, 0xc) 
  /home/cwgoes/working/go/src/github.com/cosmos/cosmos-sdk/cmd/gaia/app/invariants.go:34 +0x37e
github.com/cosmos/cosmos-sdk/cmd/gaia/app.(*GaiaApp).assertRuntimeInvariants(0xc000184d80)
  /home/cwgoes/working/go/src/github.com/cosmos/cosmos-sdk/cmd/gaia/app/invariants.go:26 +0xf6
github.com/cosmos/cosmos-sdk/cmd/gaia/app.(*GaiaApp).EndBlocker(0xc000184d80, 0xfe9600, 0xc0054feea0, 0xc0052ead00, 0xd, 0x0, 0x0, 0x0, 0x0, 0x0, ...) 
  /home/cwgoes/working/go/src/github.com/cosmos/cosmos-sdk/cmd/gaia/app/app.go:226 +0x24a
github.com/cosmos/cosmos-sdk/cmd/gaia/app.(*GaiaApp).EndBlocker-fm(0xfe9600, 0xc0054feea0, 0xc0052ead00, 0xd, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) 
  /home/cwgoes/working/go/src/github.com/cosmos/cosmos-sdk/cmd/gaia/app/app.go:173 +0xc8
github.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).EndBlock(0xc000376600, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) 
  /home/cwgoes/working/go/src/github.com/cosmos/cosmos-sdk/baseapp/baseapp.go:852 +0x105
github.com/cosmos/cosmos-sdk/x/mock/simulation.SimulateFromSeed(0xff14c0, 0xc00010ac00, 0xc000376600, 0xf150b0, 0x2c, 0xc000f580e0, 0xe, 0xe, 0xc0001c1f50, 0x5, ...) 
  /home/cwgoes/working/go/src/github.com/cosmos/cosmos-sdk/x/mock/simulation/simulate.go:184 +0x1379
github.com/cosmos/cosmos-sdk/cmd/gaia/app.TestFullGaiaSimulation(0xc00010ac00)
  /home/cwgoes/working/go/src/github.com/cosmos/cosmos-sdk/cmd/gaia/app/sim_test.go:358 +0x339
testing.tRunner(0xc00010ac00, 0xf15040)
  /usr/lib/go/src/testing/testing.go:827 +0xbf
created by testing.(*T).Run
  /usr/lib/go/src/testing/testing.go:878 +0x35c

cwgoes · 2019-02-28T17:25:14Z

All of the failing multi-seed simulations fail with loose token invariance.

cwgoes · 2019-02-28T17:37:34Z

... but that also happens on #3750

cwgoes · 2019-02-28T18:10:38Z

This was mostly a red herring stemming from a slight inaccuracy in #3750.

Dumb check

b6cb792

cwgoes mentioned this pull request Feb 27, 2019

R4R: Outstanding per-validator rewards; correctly handle same-BeginBlock redelegation-double-slash #3750

Merged

cwgoes added 2 commits February 27, 2019 17:29

Oops

797ae2b

Play with rounding, no luck yet

9be9a46

Round difference down, still no luck...

a71c095

Hook debugging, no luck

70d0724

cwgoes mentioned this pull request Feb 27, 2019

DNM: Debugging simulation failure #3746

Closed

cwgoes changed the title ~~DNM: Adhoc invariant checks~~ DNM: Adhoc invariant checks {LATEST debugging branch} Feb 27, 2019

rigelrozanski reviewed Feb 27, 2019

View reviewed changes

x/distribution/keeper/hooks.go Outdated Show resolved Hide resolved

Fix bug!

e00f61b

cwgoes added 2 commits February 28, 2019 16:21

Merge branch 'cwgoes/outstanding-per-validator' into cwgoes/adhoc-bis…

928ec7f

…ection

Fixup hook (not actually the problem)

89890f6

cwgoes changed the base branch from develop to cwgoes/outstanding-per-validator February 28, 2019 15:31

cwgoes added 3 commits February 28, 2019 16:34

Revert a few changes

b102de7

...

5ec57b5

... math bugs

f79b84a

cwgoes added 5 commits February 28, 2019 17:01

Further reduce minimal testcase

5eaf105

reduce even further

3a53dfd

... ?!

3bb24d6

Revert some changes

8c76a66

revert more changes

c2a3134

cwgoes added 4 commits February 28, 2019 17:48

Remove math checks

a5be708

Fix precision bits

439f097

typo

4ecde25

Revert invariant change

03fa195

rigelrozanski reviewed Feb 28, 2019

View reviewed changes

cwgoes closed this Feb 28, 2019

cwgoes deleted the cwgoes/adhoc-bisection branch February 28, 2019 18:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DNM: Adhoc invariant checks {LATEST debugging branch} #3759

DNM: Adhoc invariant checks {LATEST debugging branch} #3759

cwgoes commented Feb 27, 2019 •

edited

Loading

cwgoes commented Feb 27, 2019

cwgoes commented Feb 27, 2019

cwgoes commented Feb 27, 2019

cwgoes commented Feb 27, 2019

cwgoes commented Feb 27, 2019

cwgoes commented Feb 27, 2019

rigelrozanski commented Feb 28, 2019

cwgoes commented Feb 28, 2019 •

edited

Loading

cwgoes commented Feb 28, 2019

rigelrozanski Feb 28, 2019

cwgoes Feb 28, 2019

cwgoes commented Feb 28, 2019

cwgoes commented Feb 28, 2019

cwgoes commented Feb 28, 2019

cwgoes commented Feb 28, 2019

DNM: Adhoc invariant checks {LATEST debugging branch} #3759

DNM: Adhoc invariant checks {LATEST debugging branch} #3759

Conversation

cwgoes commented Feb 27, 2019 • edited Loading

cwgoes commented Feb 27, 2019

cwgoes commented Feb 27, 2019

cwgoes commented Feb 27, 2019

cwgoes commented Feb 27, 2019

cwgoes commented Feb 27, 2019

cwgoes commented Feb 27, 2019

rigelrozanski commented Feb 28, 2019

cwgoes commented Feb 28, 2019 • edited Loading

cwgoes commented Feb 28, 2019

rigelrozanski Feb 28, 2019

Choose a reason for hiding this comment

cwgoes Feb 28, 2019

Choose a reason for hiding this comment

cwgoes commented Feb 28, 2019

cwgoes commented Feb 28, 2019

cwgoes commented Feb 28, 2019

cwgoes commented Feb 28, 2019

cwgoes commented Feb 27, 2019 •

edited

Loading

cwgoes commented Feb 28, 2019 •

edited

Loading