Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'zfs send' aborts on any signal #810

Closed
DeHackEd opened this issue Jul 7, 2012 · 15 comments
Closed

'zfs send' aborts on any signal #810

DeHackEd opened this issue Jul 7, 2012 · 15 comments
Labels
Status: Inactive Not being actively updated Status: Stale No recent activity for issue Type: Defect Incorrect behavior (e.g. crash, hang)

Comments

@DeHackEd
Copy link
Contributor

DeHackEd commented Jul 7, 2012

zfs send should probably not abort when it receives a signal unless the signal is one of the default-fatal signals.

For example, start a send and pipe it however you want. Press CTRL+Z to pause. Type "fg" to resume. The send immediately aborts.

# zfs send poolname/fsname@today | bzip2 -9 > /external/poolname-today.bz2
^Z
[1]+  Stopped     zfs send poolname/fsname@today | bzip2 -9 > /external/poolname-today.bz2
# fg
zfs send poolname/fsname@today | bzip2 -9 > /external/poolname-today.bz2
warning: cannot send 'poolname/fsname@today': signal received
#

In this situation the send should probably not abort, or maybe the signal should be ignored entirely. It makes sense for other signals such as SIGTERM to cause an abort though.

@behlendorf
Copy link
Contributor

Indeed, the signal handlers seem overly broad here.

@FransUrbo
Copy link
Contributor

@behlendorf Should we add some (void) sigignore(SIGxxx); in zfs_do_send()? Which ones should we ignore in that case?

@behlendorf
Copy link
Contributor

@FransUrbo Good question. Someone would probably need to spend some time investigating that. Either we could just ignore the relevant signals or add handlers for them if we want/need to do something more clever.

@ioquatix
Copy link

I just got hit by this being half way in to what seemed like it was going to be a 60 hour transfer.. seriously this needs to be fixed or at least documented. Sending Ctrl-Z and fg/bg should do what it does for every other sane process...

@mailinglists35
Copy link

this happened to me when I attached with strace to the zfs send process. on ctrl-c to stop strace, zfs exited with message described in first post

@behlendorf behlendorf added this to the 0.7.0 milestone Mar 26, 2016
@behlendorf behlendorf modified the milestones: 0.8.0, 0.7.0 Jul 15, 2016
@jonathanvaughn
Copy link

jonathanvaughn commented Sep 20, 2016

I just ran into this today trying to use kill -STOP / kill -CONT on ZFS send.

I don't know if there's an architectural reason why this wouldn't work (i.e. zfs receive must time out if no data for X seconds), but if not, it would be nice to be able to use STOP/CONT signals to control IO load.

@DeHackEd
Copy link
Contributor Author

There is a dirty workaround: send the signal only to another process involved in the processing. If you use mbuffer, pv or similar, SIGSTOP it only.

Typically works unless you run zfs send ... | zfs receive ... since here there's nothing suitable there. So always use a buffer app. :)

@ioquatix
Copy link

This should be easy to fix.. where in the code do we need to fix it? Can someone point me in the general direction?

@jonathanvaughn
Copy link

I didn't think to try stopping pv (I was piping send | pv | receive) since I assumed send would just keep sending it data until we ran out of memory or something. That's a great work-around for now if it works (I'll try it next time I'm moving data around).

@behlendorf
Copy link
Contributor

@ioquatix it would be great if you could dig in to this. My first guess would be that zfs_ioc_send() is returning EINTR possibly do to the write() call getting interrupted in dump_record(). It looks like we might need to modify the logic right here for Linux.

Actually this may be as straight forward as fully implementing issig() in the spl. Right now it simply returns if any signal is pending and ignores the why flag which is passed in.

zfs_ioc_send()
  dmu_send_obj()
    dmu_send_impl()
      do_dump()

@jonathanvaughn
Copy link

For the record, I can confirm that kill -STOP/CONT on a process piped between send and receive (i.e. pv in my case, mbuffer, etc) works fine as a work around for now.

@behlendorf behlendorf modified the milestones: 0.8.0, 0.9.0 Oct 12, 2018
@TerraTech
Copy link
Contributor

zfs-0.7.12

Sigh - I just got bit by this tonight with a ctrl-z and then a fg to restart the process. Is there any movement on a solution being this issue is over 6 years old now. zfs send/receive should have sane handling for STOP/CONT signals.

FWIW - I would consider this a bug, because it's certainly not a feature and was surprising.

@behlendorf behlendorf removed this from the 0.9.0 milestone Nov 11, 2019
@stale
Copy link

stale bot commented Nov 11, 2020

This issue has been automatically marked as "stale" because it has not had any activity for a while. It will be closed in 90 days if no further activity occurs. Thank you for your contributions.

@stale stale bot added the Status: Stale No recent activity for issue label Nov 11, 2020
@behlendorf
Copy link
Contributor

This should be resolved in OpenZFS by #10843.

@ioquatix
Copy link

Better late than never! Well done and thank you :)

behlendorf pushed a commit that referenced this issue Apr 15, 2021
This change adds SIGSTOP and SIGTSTP handling to the issig function; 
this mirrors its behavior on Solaris. This way, long running kernel 
tasks can be stopped with the appropriate signals. Note that doing 
so with ctrl-z on the command line doesn't return control of the tty 
to the shell, because tty handling is done separately from stopping 
the process. That can be future work, if people feel that it is a 
necessary addition.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Signed-off-by: Paul Dagnelie <pcd@delphix.com>
Issue #810 
Issue #10843 
Closes #11801
behlendorf pushed a commit to behlendorf/zfs that referenced this issue Apr 21, 2021
This change adds SIGSTOP and SIGTSTP handling to the issig function; 
this mirrors its behavior on Solaris. This way, long running kernel 
tasks can be stopped with the appropriate signals. Note that doing 
so with ctrl-z on the command line doesn't return control of the tty 
to the shell, because tty handling is done separately from stopping 
the process. That can be future work, if people feel that it is a 
necessary addition.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Signed-off-by: Paul Dagnelie <pcd@delphix.com>
Issue openzfs#810 
Issue openzfs#10843 
Closes openzfs#11801
sempervictus pushed a commit to sempervictus/zfs that referenced this issue May 31, 2021
This change adds SIGSTOP and SIGTSTP handling to the issig function; 
this mirrors its behavior on Solaris. This way, long running kernel 
tasks can be stopped with the appropriate signals. Note that doing 
so with ctrl-z on the command line doesn't return control of the tty 
to the shell, because tty handling is done separately from stopping 
the process. That can be future work, if people feel that it is a 
necessary addition.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Signed-off-by: Paul Dagnelie <pcd@delphix.com>
Issue openzfs#810 
Issue openzfs#10843 
Closes openzfs#11801
tonyhutter pushed a commit to tonyhutter/zfs that referenced this issue Sep 7, 2021
This change adds SIGSTOP and SIGTSTP handling to the issig function;
this mirrors its behavior on Solaris. This way, long running kernel
tasks can be stopped with the appropriate signals. Note that doing
so with ctrl-z on the command line doesn't return control of the tty
to the shell, because tty handling is done separately from stopping
the process. That can be future work, if people feel that it is a
necessary addition.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Signed-off-by: Paul Dagnelie <pcd@delphix.com>
Issue openzfs#810
Issue openzfs#10843
Closes openzfs#11801
tonyhutter pushed a commit that referenced this issue Sep 21, 2021
This change adds SIGSTOP and SIGTSTP handling to the issig function;
this mirrors its behavior on Solaris. This way, long running kernel
tasks can be stopped with the appropriate signals. Note that doing
so with ctrl-z on the command line doesn't return control of the tty
to the shell, because tty handling is done separately from stopping
the process. That can be future work, if people feel that it is a
necessary addition.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Signed-off-by: Paul Dagnelie <pcd@delphix.com>
Issue #810
Issue #10843
Closes #11801
tonyhutter pushed a commit that referenced this issue Sep 22, 2021
This change adds SIGSTOP and SIGTSTP handling to the issig function;
this mirrors its behavior on Solaris. This way, long running kernel
tasks can be stopped with the appropriate signals. Note that doing
so with ctrl-z on the command line doesn't return control of the tty
to the shell, because tty handling is done separately from stopping
the process. That can be future work, if people feel that it is a
necessary addition.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Signed-off-by: Paul Dagnelie <pcd@delphix.com>
Issue #810
Issue #10843
Closes #11801
ghost pushed a commit to truenas/zfs that referenced this issue Oct 7, 2021
This change adds SIGSTOP and SIGTSTP handling to the issig function;
this mirrors its behavior on Solaris. This way, long running kernel
tasks can be stopped with the appropriate signals. Note that doing
so with ctrl-z on the command line doesn't return control of the tty
to the shell, because tty handling is done separately from stopping
the process. That can be future work, if people feel that it is a
necessary addition.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Signed-off-by: Paul Dagnelie <pcd@delphix.com>
Issue openzfs#810
Issue openzfs#10843
Closes openzfs#11801
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Status: Inactive Not being actively updated Status: Stale No recent activity for issue Type: Defect Incorrect behavior (e.g. crash, hang)
Projects
None yet
Development

No branches or pull requests

8 participants
@ioquatix @behlendorf @FransUrbo @TerraTech @DeHackEd @jonathanvaughn @mailinglists35 and others