Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make threadcall gc safe #55956

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Make threadcall gc safe #55956

wants to merge 4 commits into from

Conversation

gbaraldi
Copy link
Member

@gbaraldi gbaraldi commented Oct 1, 2024

threadcall cannot call julia code. But due to thread adoption it currently is not marked gc safe. Which means that a blocking call will cause a hang in the GC

@gbaraldi gbaraldi requested a review from vtjnash October 1, 2024 14:18
@gbaraldi gbaraldi added backport 1.10 Change should be backported to the 1.10 release backport 1.11 Change should be backported to release-1.11 labels Oct 1, 2024
base/threadcall.jl Outdated Show resolved Hide resolved
@vchuravy
Copy link
Member

vchuravy commented Oct 1, 2024

Someone should finish #49933

@JamesWrigley
Copy link
Contributor

Just to confirm, is it safe to call jl_gc_safe_{enter/leave}() within a GC safepoint? Asking because I'll probably keep my hack of manually calling it to support current/older Julia versions.

@vchuravy
Copy link
Member

vchuravy commented Oct 1, 2024

"GC safepoint" are singular points in time.

GC safe regions are spans in time, and yes it is legal to to nest safe regions.

@@ -43,7 +43,9 @@ macro threadcall(f, rettype, argtypes, argvals...)
push!(body, :(p += Core.sizeof($T)))
push!(args, arg)
end
push!(body, :(gc_state = ccall(:jl_gc_safe_enter, Int8, (), )))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So a problem with this is that the ccall lowering will insert calls to cconvert and then unsafe_convert. cconvert is designed to allocate.

So you manually need to insert the code to lower include aGC.@preserve

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have I told you how much I dislike threadcall?

Copy link
Member

@maleadt maleadt Oct 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also see JuliaInterop/ObjectiveC.jl@2578dc2, which is even more relevant (it turns a ccall into a GC-safe ccall after manually doing argument conversion first). It would almost be easier to fix #49933 at this point...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue is that we need to backport this :(

@@ -39,13 +39,19 @@ macro threadcall(f, rettype, argtypes, argvals...)
args = Symbol[]
for (i, T) in enumerate(argtypes)
arg = Symbol("arg", i)
push!(body, :($arg = unsafe_load(convert(Ptr{$T}, p))))
push!(body, :($arg = unsafe_convert($T, cconvert($T, unsafe_load(convert(Ptr{$T}, p))))))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't fully understand the code here (why is there an unsafe_load), but the return from cconvert must be GC preserved.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Threadcall passes the args in an array of ptrs if I understood the code correctly

push!(body, :(unsafe_store!(convert(Ptr{$rettype}, retval_ptr), ret)))
push!(body, :(return Int(Core.sizeof($rettype))))
append!(body, (quote
GC.@preserve $(args...) begin
Copy link
Member

@vchuravy vchuravy Oct 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are preserving the unsafe_convert not the cconvert here, right?

@KristofferC KristofferC mentioned this pull request Oct 18, 2024
43 tasks
@KristofferC KristofferC mentioned this pull request Oct 29, 2024
47 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 1.10 Change should be backported to the 1.10 release backport 1.11 Change should be backported to release-1.11
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants