ZTS: Fix zpool_reopen_001_pos #9680
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
behlendorf
added
Component: Test Suite
Update the vdev_disk_open() retry logic to use a specified number of milliseconds to be more robust. Additionally, on failure log both the time waited and requested timeout to the internal log. The default maximum allowed open retry time has been increased from 500ms to 1000ms. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
behlendorf
force-pushed
the
zpool_reopen_001_pos
branch
from
f76f598 to
f63b930
Compare
|
With the debugging from the PR and the CI logs I was able to verify that the 500ms timeout was in fact reached. Given that, I've increased the timeout to 1000ms to see if the issue can still be reproduced. |
Codecov Report
@@ Coverage Diff @@
## master #9680 +/- ##
========================================
+ Coverage 79% 79% +<1%
========================================
Files 418 418
Lines 123572 123575 +3
========================================
+ Hits 97956 98193 +237
+ Misses 25616 25382 -234
Continue to review full report at Codecov.
|
PrivatePuffin
approved these changes
Dec 8, 2019
|
@behlendorf If you mark it "not draft, ready for review as real PR" maybe remove the "WIP" tag and replace it with "review requested"? |
behlendorf
added
Status: Code Review Needed
behlendorf
changed the title
|
Doubling the timeout appears to have resolved the issue. After running this PR through many times I haven't see this failure. At a minimum it does appear to reduce the frequency. |
behlendorf
added
Status: Accepted
tonyhutter
pushed a commit
to tonyhutter/zfs
that referenced
this pull request
Dec 26, 2019
Update the vdev_disk_open() retry logic to use a specified number of milliseconds to be more robust. Additionally, on failure log both the time waited and requested timeout to the internal log. The default maximum allowed open retry time has been increased from 500ms to 1000ms. Reviewed-by: Kjeld Schouten <kjeld@schouten-lebbing.nl> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes openzfs#9680 Conflicts:
tonyhutter
pushed a commit
to tonyhutter/zfs
that referenced
this pull request
Dec 27, 2019
Update the vdev_disk_open() retry logic to use a specified number of milliseconds to be more robust. Additionally, on failure log both the time waited and requested timeout to the internal log. The default maximum allowed open retry time has been increased from 500ms to 1000ms. Reviewed-by: Kjeld Schouten <kjeld@schouten-lebbing.nl> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes openzfs#9680 Conflicts:
tonyhutter
pushed a commit
that referenced
this pull request
Jan 23, 2020
Update the vdev_disk_open() retry logic to use a specified number of milliseconds to be more robust. Additionally, on failure log both the time waited and requested timeout to the internal log. The default maximum allowed open retry time has been increased from 500ms to 1000ms. Reviewed-by: Kjeld Schouten <kjeld@schouten-lebbing.nl> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #9680 Conflicts:
shartse
pushed a commit
to shartse/zfs
that referenced
this pull request
Jul 21, 2020
Update the vdev_disk_open() retry logic to use a specified number of milliseconds to be more robust. Additionally, on failure log both the time waited and requested timeout to the internal log. The default maximum allowed open retry time has been increased from 500ms to 1000ms. Reviewed-by: Kjeld Schouten <kjeld@schouten-lebbing.nl> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes openzfs#9680
shartse
pushed a commit
to shartse/zfs
that referenced
this pull request
Aug 19, 2020
Update the vdev_disk_open() retry logic to use a specified number of milliseconds to be more robust. Additionally, on failure log both the time waited and requested timeout to the internal log. The default maximum allowed open retry time has been increased from 500ms to 1000ms. Reviewed-by: Kjeld Schouten <kjeld@schouten-lebbing.nl> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes openzfs#9680
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation and Context
Debug CI failures of zpool_reopen_001_pos. I wasn't able to reproduce
this locally so I've updated the code to provide additional debugging.
http://build.zfsonlinux.org/builders/Ubuntu%2018.04%20x86_64%20%28TEST%29/builds/6326
There default timeout values have not been yet changed since I'd like to
reproduce this issue if possible.
note: Hitting the open timeout has only been observed with test that
use a scsi_debug device. Other occasional test failures appear to be
encountering the same issue.
Description
Update the vdev_disk_open() retry logic to use a specified number
of milliseconds to be more robust. Additionally, log both the
time waited and requested timeout to the internal log for debugging.
How Has This Been Tested?
Locally by running the
zpool_reopenZTS tests. Pending full CI run.Types of changes
Checklist:
Signed-off-by.