Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test case: rsend_019_pos #6086

Closed
bunder2015 opened this issue May 1, 2017 · 2 comments
Closed

Test case: rsend_019_pos #6086

bunder2015 opened this issue May 1, 2017 · 2 comments
Labels
Component: Test Suite Indicates an issue with the test framework or a test case Status: Stale No recent activity for issue

Comments

@bunder2015
Copy link
Contributor

bunder2015 commented May 1, 2017 •
edited
Loading

System information

Type Version/Name
Distribution Name Gentoo
Distribution Version
Linux Kernel 4.4.52-gentoo
Architecture amd64 (intel haswell)
ZFS Version 0.7.0-rc3 (git head + #6074)
SPL Version 0.7.0-rc3 (git head)

Describe the problem you're observing

rsend_019_pos occasionally takes way way longer than it should, and eventually gets killed off by the test scripts. Average run time is less than a minute, sometimes can run longer than 45 minutes on my VM despite the 10 minute cutoff timer. Culprit appears to be a zfs recv operation. ZFS version doesn't seem to matter much, anything recent seems to exhibit the same behaviour.

Test: /home/testuser/zfs/tests/zfs-tests/tests/functional/rsend/rsend_019_pos (run as root) [46:49] [KILLED]

Describe how to reproduce the problem

Usually can get triggered by running tests repeatedly on the same system without rebooting, doesn't appear to happen on kernel 4.9, but it does on 4.4. Seems to happen on occasion to the Ubuntu 14.04 i686 testbot recently as well. eg1 eg2 eg3

Include any warning/errors/backtraces from the system logs

I grabbed these stack traces while the test was spinning its wheels, I hope they are useful.

zfs.16456
txg_quiesce.28243
txg_sync.28244
txg_quiesce.28087
txg_sync.28088
receive_writer.16519
@loli10K loli10K added the Component: Test Suite Indicates an issue with the test framework or a test case label May 1, 2017
behlendorf added a commit to behlendorf/zfs that referenced this issue May 1, 2017
@behlendorf
Disable 3 test cases
e165357 
The following three test cases occasionally fail the automated
testing.  For the moment disable these test cases until the
underlying issues can be fully investigated and resolved.

zfs_destroy_010_pos - openzfs#5893
rsend_020_pos       - openzfs#6086 (32-bit only)
send-c_volume       - openzfs#6087 (32-bit only)

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
behlendorf added a commit to behlendorf/zfs that referenced this issue May 1, 2017
@behlendorf
Disable 3 test cases
d46e722 
The following three test cases occasionally fail the automated
testing.  For the moment disable these test cases until the
underlying issues can be fully investigated and resolved.

zfs_destroy_010_pos - openzfs#5893
rsend_020_pos       - openzfs#6086 (32-bit only)
send-c_volume       - openzfs#6087 (32-bit only)

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#5893
Issue openzfs#6086
Issue openzfs#6087
@behlendorf behlendorf mentioned this issue May 1, 2017
Merged
@behlendorf
Copy link
Contributor

@bunder2015 thanks for filing this. We've definitely been seeing this for a while on master but haven't had a chance to run it down.

behlendorf added a commit to behlendorf/zfs that referenced this issue May 4, 2017
@behlendorf
Disable rsend_020_pos and send-c_volume test cases
a4f0bb9 
The following three test cases occasionally fail the automated
testing.  For the moment disable these test cases until the
underlying issues can be fully investigated and resolved.

rsend_020_pos       - openzfs#6086 (32-bit only)
send-c_volume       - openzfs#6087 (32-bit only)

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#6086
Issue openzfs#6087
behlendorf added a commit to behlendorf/zfs that referenced this issue May 4, 2017
@behlendorf
Disable rsend_019_pos and send-c_volume test cases
da34f7c 
The following three test cases occasionally fail the automated
testing.  For the moment disable these test cases until the
underlying issues can be fully investigated and resolved.

rsend_019_pos       - openzfs#6086 (32-bit only)
send-c_volume       - openzfs#6087 (32-bit only)

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#6086
Issue openzfs#6087
behlendorf pushed a commit that referenced this issue Sep 25, 2017
@loli10K @behlendorf
Fix some ZFS Test Suite issues
3fd3e56 
* Add 'zfs bookmark' coverage (zfs_bookmark_cliargs)

 * Add OpenZFS 8166 coverage (zpool_scrub_offline_device)

 * Fix "busy" zfs_mount_remount failures

 * Fix bootfs_003_pos, bootfs_004_neg, zdb_005_pos local cleanup

 * Update usage of $KEEP variable, add get_all_pools() function

 * Enable history_008_pos and rsend_019_pos (non-32bit builders)

 * Enable zfs_copies_005_neg, update local cleanup

 * Fix zfs_send_007_pos (large_dnode + OpenZFS 8199)

 * Fix rollback_003_pos (use dataset name, not mountpoint, to unmount)

 * Update default_raidz_setup() to work properly with more than 3 disks

 * Use $TEST_BASE_DIR instead of hardcoded (/var)/tmp for file VDEVs

 * Update usage of /dev/random to /dev/urandom

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: loli10K <ezomori.nozomu@gmail.com>
Issue #6086 
Closes #5658 
Closes #6143 
Closes #6421 
Closes #6627 
Closes #6632
@Nasf-Fan Nasf-Fan mentioned this issue Oct 10, 2017
Merged
13 tasks
FransUrbo pushed a commit to FransUrbo/zfs that referenced this issue Apr 28, 2019
@loli10K @FransUrbo
Fix some ZFS Test Suite issues
25c19d7 
* Add 'zfs bookmark' coverage (zfs_bookmark_cliargs)

 * Add OpenZFS 8166 coverage (zpool_scrub_offline_device)

 * Fix "busy" zfs_mount_remount failures

 * Fix bootfs_003_pos, bootfs_004_neg, zdb_005_pos local cleanup

 * Update usage of $KEEP variable, add get_all_pools() function

 * Enable history_008_pos and rsend_019_pos (non-32bit builders)

 * Enable zfs_copies_005_neg, update local cleanup

 * Fix zfs_send_007_pos (large_dnode + OpenZFS 8199)

 * Fix rollback_003_pos (use dataset name, not mountpoint, to unmount)

 * Update default_raidz_setup() to work properly with more than 3 disks

 * Use $TEST_BASE_DIR instead of hardcoded (/var)/tmp for file VDEVs

 * Update usage of /dev/random to /dev/urandom

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: loli10K <ezomori.nozomu@gmail.com>
Issue openzfs#6086 
Closes openzfs#5658 
Closes openzfs#6143 
Closes openzfs#6421 
Closes openzfs#6627 
Closes openzfs#6632
@stale
Copy link

stale bot commented Aug 25, 2020

This issue has been automatically marked as "stale" because it has not had any activity for a while. It will be closed in 90 days if no further activity occurs. Thank you for your contributions.

@stale stale bot added the Status: Stale No recent activity for issue label Aug 25, 2020
@stale stale bot closed this as completed Nov 23, 2020
behlendorf added a commit that referenced this issue Dec 21, 2021
@behlendorf
ZTS: rsend_007_pos failures
7b5d783 
The rsend_007_pos test reliably fails on Linux in the cleanup
function.  This is caused by an unmount error when attempting to
recursively destroy the newly received datasets.  Invoking `df`
prior to the `zfs destroy` interestingly avoids the unmont error.

Why this should matter is unclear and should be investigated.
However, this minor tweak may allow us to remove the ZTS rsend
exceptions.  The subsequent rsend_010_pos and rsend_011_pos
failures were a result of this initial failure.  The other
"maybe" failures I was unable to reproduce and have not been
recently observed in the master branch.

Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #5665
Closes #6086
Closes #6087
Closes #6446
Closes #12876
tonyhutter pushed a commit to tonyhutter/zfs that referenced this issue Feb 10, 2022
@behlendorf @tonyhutter
ZTS: rsend_007_pos failures
fa93a7d 
The rsend_007_pos test reliably fails on Linux in the cleanup
function.  This is caused by an unmount error when attempting to
recursively destroy the newly received datasets.  Invoking `df`
prior to the `zfs destroy` interestingly avoids the unmont error.

Why this should matter is unclear and should be investigated.
However, this minor tweak may allow us to remove the ZTS rsend
exceptions.  The subsequent rsend_010_pos and rsend_011_pos
failures were a result of this initial failure.  The other
"maybe" failures I was unable to reproduce and have not been
recently observed in the master branch.

Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes openzfs#5665
Closes openzfs#6086
Closes openzfs#6087
Closes openzfs#6446
Closes openzfs#12876
tonyhutter pushed a commit to tonyhutter/zfs that referenced this issue Feb 14, 2022
@behlendorf @tonyhutter
ZTS: rsend_007_pos failures
3013a15 
The rsend_007_pos test reliably fails on Linux in the cleanup
function.  This is caused by an unmount error when attempting to
recursively destroy the newly received datasets.  Invoking `df`
prior to the `zfs destroy` interestingly avoids the unmont error.

Why this should matter is unclear and should be investigated.
However, this minor tweak may allow us to remove the ZTS rsend
exceptions.  The subsequent rsend_010_pos and rsend_011_pos
failures were a result of this initial failure.  The other
"maybe" failures I was unable to reproduce and have not been
recently observed in the master branch.

Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes openzfs#5665
Closes openzfs#6086
Closes openzfs#6087
Closes openzfs#6446
Closes openzfs#12876
tonyhutter pushed a commit to tonyhutter/zfs that referenced this issue Feb 16, 2022
@behlendorf @tonyhutter
ZTS: rsend_007_pos failures
200c05d 
The rsend_007_pos test reliably fails on Linux in the cleanup
function.  This is caused by an unmount error when attempting to
recursively destroy the newly received datasets.  Invoking `df`
prior to the `zfs destroy` interestingly avoids the unmont error.

Why this should matter is unclear and should be investigated.
However, this minor tweak may allow us to remove the ZTS rsend
exceptions.  The subsequent rsend_010_pos and rsend_011_pos
failures were a result of this initial failure.  The other
"maybe" failures I was unable to reproduce and have not been
recently observed in the master branch.

Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes openzfs#5665
Closes openzfs#6086
Closes openzfs#6087
Closes openzfs#6446
Closes openzfs#12876
tonyhutter pushed a commit to tonyhutter/zfs that referenced this issue Feb 17, 2022
@behlendorf @tonyhutter
ZTS: rsend_007_pos failures
be01ee8 
The rsend_007_pos test reliably fails on Linux in the cleanup
function.  This is caused by an unmount error when attempting to
recursively destroy the newly received datasets.  Invoking `df`
prior to the `zfs destroy` interestingly avoids the unmont error.

Why this should matter is unclear and should be investigated.
However, this minor tweak may allow us to remove the ZTS rsend
exceptions.  The subsequent rsend_010_pos and rsend_011_pos
failures were a result of this initial failure.  The other
"maybe" failures I was unable to reproduce and have not been
recently observed in the master branch.

Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes openzfs#5665
Closes openzfs#6086
Closes openzfs#6087
Closes openzfs#6446
Closes openzfs#12876
nicman23 pushed a commit to nicman23/zfs that referenced this issue Aug 22, 2022
@behlendorf @nicman23
ZTS: rsend_007_pos failures
1d39d5d 
The rsend_007_pos test reliably fails on Linux in the cleanup
function.  This is caused by an unmount error when attempting to
recursively destroy the newly received datasets.  Invoking `df`
prior to the `zfs destroy` interestingly avoids the unmont error.

Why this should matter is unclear and should be investigated.
However, this minor tweak may allow us to remove the ZTS rsend
exceptions.  The subsequent rsend_010_pos and rsend_011_pos
failures were a result of this initial failure.  The other
"maybe" failures I was unable to reproduce and have not been
recently observed in the master branch.

Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes openzfs#5665
Closes openzfs#6086
Closes openzfs#6087
Closes openzfs#6446
Closes openzfs#12876
nicman23 pushed a commit to nicman23/zfs that referenced this issue Aug 22, 2022
@behlendorf @nicman23
ZTS: rsend_007_pos failures
4c55ec0 
The rsend_007_pos test reliably fails on Linux in the cleanup
function.  This is caused by an unmount error when attempting to
recursively destroy the newly received datasets.  Invoking `df`
prior to the `zfs destroy` interestingly avoids the unmont error.

Why this should matter is unclear and should be investigated.
However, this minor tweak may allow us to remove the ZTS rsend
exceptions.  The subsequent rsend_010_pos and rsend_011_pos
failures were a result of this initial failure.  The other
"maybe" failures I was unable to reproduce and have not been
recently observed in the master branch.

Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes openzfs#5665
Closes openzfs#6086
Closes openzfs#6087
Closes openzfs#6446
Closes openzfs#12876
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Component: Test Suite Indicates an issue with the test framework or a test case Status: Stale No recent activity for issue
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@behlendorf @loli10K @bunder2015