Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix IO deadlock condition #22886

Merged
merged 1 commit into from
Jul 28, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 7 additions & 5 deletions src/jl_uv.c
Original file line number Diff line number Diff line change
Expand Up @@ -197,16 +197,18 @@ JL_DLLEXPORT void jl_close_uv(uv_handle_t *handle)
}

if (handle->type == UV_NAMED_PIPE || handle->type == UV_TCP) {
uv_stream_t *stream = (uv_stream_t*)handle;
#ifdef _OS_WINDOWS_
if (((uv_stream_t*)handle)->stream.conn.shutdown_req) {
if (stream->stream.conn.shutdown_req) {
#else
if (((uv_stream_t*)handle)->shutdown_req) {
if (stream->shutdown_req) {
#endif
// don't close the stream while attempting a graceful shutdown
return;
}
if (uv_is_writable((uv_stream_t*)handle)) {
if (uv_is_writable(stream) && stream->write_queue_size != 0) {
// attempt graceful shutdown of writable streams to give them a chance to flush first
// TODO: introduce a uv_drain cb API instead of abusing uv_shutdown in this way
uv_shutdown_t *req = (uv_shutdown_t*)malloc(sizeof(uv_shutdown_t));
req->data = 0;
/*
Expand All @@ -218,12 +220,12 @@ JL_DLLEXPORT void jl_close_uv(uv_handle_t *handle)
* b) In case the stream is already closed, in which case uv_close would
* cause an assertion failure.
*/
uv_shutdown(req, (uv_stream_t*)handle, &jl_uv_shutdownCallback);
uv_shutdown(req, stream, &jl_uv_shutdownCallback);
return;
}
}

if (!uv_is_closing((uv_handle_t*)handle)) {
if (!uv_is_closing(handle)) {
// avoid double-closing the stream
if (handle->type == UV_TTY)
uv_tty_set_mode((uv_tty_t*)handle, UV_TTY_MODE_NORMAL);
Expand Down
34 changes: 30 additions & 4 deletions test/spawn.jl
Original file line number Diff line number Diff line change
Expand Up @@ -298,15 +298,25 @@ let out = Pipe(), echo = `$exename --startup-file=no -e 'print(STDOUT, " 1\t", r
@test iswritable(out)
close(out.in)
@test !isopen(out.in)
Sys.iswindows() || @test !isopen(out.out) # it takes longer to propagate EOF through the Windows event system
@test_throws ArgumentError write(out, "now closed error")
@test isreadable(out)
@test !iswritable(out)
if !Sys.iswindows()
# on UNIX, we expect the pipe buffer is big enough that the write queue was immediately emptied
# and so we should already be notified of EPIPE on out.out by now
# and the other task should have already managed to consume all of the output
# it takes longer to propagate EOF through the Windows event system
# since it appears to be unwilling to buffer as much data
@test !isopen(out.out)
@test !isreadable(out)
end
@test_throws ArgumentError write(out, "now closed error")
if Sys.iswindows()
# WINNT kernel does not provide a fast mechanism for async propagation
# WINNT kernel appears to not provide a fast mechanism for async propagation
# of EOF for a blocking stream, so just wait for it to catch up.
# This shouldn't take much more than 32ms.
Base.wait_close(out)
# it's closed now, but the other task is expected to be behind this task
# in emptying the read buffer
@test isreadable(out)
end
@test !isopen(out)
end
Expand Down Expand Up @@ -469,3 +479,19 @@ let c = `ls -l "foo bar"`
@test length(c) == 3
@test eachindex(c) == 1:3
end

## Deadlock in spawning a cmd (#22832)
# FIXME?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what was broken about this? use test_broken

Copy link
Contributor

@tkelman tkelman Jul 28, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hello?

PLEASE communicate more here. No one can read your mind. What didn't work about this, a FIXME comment with no details and a commented out test is better than nothing, but doesn't actually help anyone figure out what is wrong here and why the test doesn't work, can't use @test_broken, or the fix is incomplete. You do this over and over and over again and I implore you every time to cut it out.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mentioned in the merge commit text. I don't know the answers to any of those questions, so I can't provide more information. If I knew any of those answers, I would have already fixed the test.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would test_broken have worked? A simple response to the question "it froze on CI and couldn't reproduce locally" would be a lot more visible than the merge commit text.

#let stdout = Pipe(), stdin = Pipe()
# Base.link_pipe(stdout, julia_only_read=true)
# Base.link_pipe(stdin, julia_only_write=true)
# p = spawn(pipeline(catcmd, stdin=stdin, stdout=stdout, stderr=DevNull))
# @async begin # feed cat with 2 MB of data (zeros)
# write(stdin, zeros(UInt8, 1048576 * 2))
# close(stdin)
# end
# sleep(0.5) # give cat a chance to fill the write buffer for stdout
# close(stdout.in) # make sure we can still close the write end
# @test sizeof(readstring(stdout)) == 1048576 * 2 # make sure we get all the data
# @test success(p)
#end