Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

teamcity: failed test: bank/node-restart #30628

Closed
cockroach-teamcity opened this issue Sep 25, 2018 · 5 comments
Closed

teamcity: failed test: bank/node-restart #30628

cockroach-teamcity opened this issue Sep 25, 2018 · 5 comments
Assignees
Labels
A-kv-client Relating to the KV client and the KV interface. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot.
Milestone

Comments

@cockroach-teamcity
Copy link
Member

The following tests appear to have failed on master (roachtest): acceptance/bank/node-restart

You may want to check for open issues.

#924701:

acceptance/bank/node-restart
--- FAIL: roachtest/acceptance/bank/node-restart (600.126s)
	test.go:779: test timed out (10m0s)
	test.go:498,bank.go:276,bank.go:402,acceptance.go:60: context canceled





Please assign, take a look and update the issue accordingly.

@cockroach-teamcity cockroach-teamcity added this to the 2.2 milestone Sep 25, 2018
@cockroach-teamcity cockroach-teamcity added C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. labels Sep 25, 2018
@tbg
Copy link
Member

tbg commented Sep 25, 2018

@petermattis my reading of this is that the test timed out?

If I remember correctly this test was always flaky under shutdown. Except back then we got a goroutine dump -- we should bring that back, WDYT? None of the artifacts right now are helpful in any way.

@tbg tbg added the A-kv-client Relating to the KV client and the KV interface. label Sep 25, 2018
@tbg tbg modified the milestones: 2.2, 1.1.x, 2.1 Sep 25, 2018
@petermattis
Copy link
Collaborator

I don't recall the previous goroutine dump. Were we grabbing goroutines from each of the nodes?

@tbg
Copy link
Member

tbg commented Sep 25, 2018

No, the test would time out and the Go test runner would dump all goroutines, which in this case includes the chaos monkey. It wouldn't tell us about the server side of things (but the problem is likely in the chaos monkey).

@tbg
Copy link
Member

tbg commented Sep 25, 2018

(but destroying the cluster with -SIGQUIT to get goroutines of the nodes is a good idea too)

@tbg
Copy link
Member

tbg commented Sep 26, 2018

Closing for #30064, even if it turns out to be a different failure mode.

@tbg tbg closed this as completed Sep 26, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-kv-client Relating to the KV client and the KV interface. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot.
Projects
None yet
Development

No branches or pull requests

3 participants