Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

storage: mvccstats inconsistencies during tpcc #28946

Closed
tbg opened this issue Aug 22, 2018 · 1 comment
Closed

storage: mvccstats inconsistencies during tpcc #28946

tbg opened this issue Aug 22, 2018 · 1 comment
Labels
A-kv-client Relating to the KV client and the KV interface. C-investigation Further steps needed to qualify. C-label will change.
Milestone

Comments

@tbg
Copy link
Member

tbg commented Aug 22, 2018

Running ./cockroach debug check-store against nodes from #28918, I see mismatches like this:

Note in particular r137 below, whose stats are pure garbage.
At first glance, it looks like this might be spillover from the timeseries ranges (which have approximate stats that tend to go negative when GC happens) but we're supposedly tracking the ranges which hold estimates and the ranges below claim not to.

We should generally be able assert that there isn't a stats mismatch and do so in all roachtests at opportune times.

r127:/Table/54/1/9{27/3/-2045/0-64/2/-1444} [(n1,s1):1, (n3,s3):2, (n2,s2):3, next=4, gen=842355161184]: diff(actual, claimed): GCBytesAge: 9090058794 != 9427657411
LiveBytes: 48418643 != 48349513
LiveCount: 1117871 != 1116225
KeyBytes: 25796071 != 25777965
KeyCount: 1117871 != 1116225

r402:/Table/54/{1/964/2/-1444/0-2} [(n1,s1):1, (n3,s3):2, (n2,s2):3, next=4, gen=842353299136]: diff(actual, claimed): GCBytesAge: 8582510619 != 9048690353
LiveBytes: 47015452 != 46930069
LiveCount: 1085445 != 1083412
KeyBytes: 25046213 != 25023850
KeyCount: 1085445 != 1083412

r137:/Table/55/1/{0/0-1000} [(n1,s1):1, (n3,s3):2, (n2,s2):3, next=4, gen=842350802872]: diff(actual, claimed): GCBytesAge: 459022290695 != 468795486289
LiveBytes: 82579 != -846349
LiveCount: 1000 != -12823
KeyBytes: 4112650 != 4021050
KeyCount: 1000 != -12823

r203:/Table/56/1/{0/1/2101/0-138/8/2871} [(n1,s1):1, (n3,s3):2, (n2,s2):3, next=4, gen=842354018792]: diff(actual, claimed): GCBytesAge: 75133733043 != 75467250960
LiveBytes: 32065065 != 31993943
LiveCount: 1222845 != 1220121
KeyBytes: 29396707 != 29371893
KeyCount: 1327389 != 1324665

r5553:/Table/56/1/{138/8/2871-272/10/2564} [(n1,s1):1, (n3,s3):2, (n2,s2):3, next=4, gen=0]: diff(actual, claimed): GCBytesAge: 40028914124 != 40406514945
LiveBytes: 32306114 != 32213674
LiveCount: 1190224 != 1186820
KeyBytes: 28209986 != 28175414
KeyCount: 1245160 != 1241756
@tbg tbg added the A-kv-client Relating to the KV client and the KV interface. label Aug 22, 2018
@tbg tbg added this to the 2.2 milestone Aug 22, 2018
@knz knz added the C-investigation Further steps needed to qualify. C-label will change. label Sep 21, 2018
@tbg
Copy link
Member Author

tbg commented Apr 25, 2019

We've since fixed multiple inconsistencies that would result in such negative stats being generated, see for instance #36861.

@tbg tbg closed this as completed Apr 25, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-kv-client Relating to the KV client and the KV interface. C-investigation Further steps needed to qualify. C-label will change.
Projects
None yet
Development

No branches or pull requests

2 participants