reset `RandomDevice` file from `init` #42537

JeffBezanson · 2021-10-07T20:32:54Z

This prevents us from seeing an invalid IOStream object from a saved system image, and also ensures the files are initialized once for all threads.

We don't necessarily need both files, but eagerly initializing them both is much simpler than any alternative I can think of.

KristofferC · 2021-10-09T15:00:45Z

stdlib/Random/src/RNGs.jl

+        DEV_RANDOM[] = try
+            open("/dev/random")
+        catch
+            nothing


Why not set them unconditionally to nothing and let them be opened when they are needed?

That would be a race condition. I guess it's ok if we're willing to leak nthreads file descriptors.

Yeah, that's what the current code has as well, right? But might as well go with this, shouldn't really have a measurable startup cost.

You can now declare it @atomic, and use replacefield!(nothing => fd), and then close it if you lost the race. Eventually we will likely want to add a Once-like object, which runs a thunk exactly once.

That would be good; are we ok with not backporting to 1.6?

This prevents us from seeing an invalid `IOStream` object from a saved system image, and also ensures the files are opened once for all threads.

KristofferC · 2021-10-22T18:48:34Z

Rebased to hopefully improve CI.

This prevents us from seeing an invalid `IOStream` object from a saved system image, and also ensures the files are opened once for all threads.

This prevents us from seeing an invalid `IOStream` object from a saved system image, and also ensures the files are opened once for all threads. (cherry picked from commit c74814e)

commit c054dbc Author: Shuhei Kadowaki <40514306+aviatesk@users.noreply.github.com> Date: Fri Oct 29 01:31:55 2021 +0900 optimizer: eliminate allocations (JuliaLang#42833) commit 6a9737d Author: Jeff Bezanson <jeff.bezanson@gmail.com> Date: Thu Oct 28 12:23:53 2021 -0400 fix JuliaLang#42659, move `jl_coverage_visit_line` to runtime library (JuliaLang#42810) commit c762f10 Author: Marc Ittel <35898736+MarcMush@users.noreply.github.com> Date: Thu Oct 28 12:19:13 2021 +0200 change `julia` to `julia-repl` in docstrings (JuliaLang#42824) Co-authored-by: Michael Abbott <32575566+mcabbott@users.noreply.github.com> commit 9f52ec0 Author: Dilum Aluthge <dilum@aluthge.com> Date: Thu Oct 28 05:30:11 2021 -0400 CI (Buildkite): Update all rootfs images to the latest versions (JuliaLang#42802) * CI (Buildkite): Update all rootfs images to the latest versions * Re-sign all of the signed pipelines commit 404e584 Author: DilumAluthgeBot <43731525+DilumAluthgeBot@users.noreply.github.com> Date: Wed Oct 27 21:11:04 2021 -0400 🤖 Bump the Statistics stdlib from 74897fe to 5256d57 (JuliaLang#42826) Co-authored-by: Dilum Aluthge <dilum@aluthge.com> commit c74814e Author: Jeff Bezanson <jeff.bezanson@gmail.com> Date: Wed Oct 27 16:34:46 2021 -0400 reset `RandomDevice` file from `__init__` (JuliaLang#42537) This prevents us from seeing an invalid `IOStream` object from a saved system image, and also ensures the files are opened once for all threads. commit 05ed348 Author: Jeff Bezanson <jeff.bezanson@gmail.com> Date: Wed Oct 27 15:24:17 2021 -0400 only visit nonfunction_mt once when traversing method tables (JuliaLang#42821) commit d71b77d Author: DilumAluthgeBot <43731525+DilumAluthgeBot@users.noreply.github.com> Date: Tue Oct 26 20:39:08 2021 -0400 🤖 Bump the Downloads stdlib from 5f1509d to dbb0625 (JuliaLang#42811) Co-authored-by: Dilum Aluthge <dilum@aluthge.com> commit b4fddc1 Author: DilumAluthgeBot <43731525+DilumAluthgeBot@users.noreply.github.com> Date: Tue Oct 26 14:46:20 2021 -0400 🤖 Bump the Pkg stdlib from bc32103f to 26918395 (JuliaLang#42806) Co-authored-by: Dilum Aluthge <dilum@aluthge.com> commit 6a386de Author: Dilum Aluthge <dilum@aluthge.com> Date: Tue Oct 26 12:15:51 2021 -0400 CI (Buildkite): make sure to hit ignore any unencrypted repo keys, regardless of where they are located in the repository (JuliaLang#42803) commit 021a6b5 Author: Shuhei Kadowaki <40514306+aviatesk@users.noreply.github.com> Date: Wed Oct 27 01:08:33 2021 +0900 optimizer: clean up inlining test code (JuliaLang#42804) commit 16eb196 Merge: 21ebabf 1510eaa Author: Shuhei Kadowaki <40514306+aviatesk@users.noreply.github.com> Date: Tue Oct 26 23:25:41 2021 +0900 Merge pull request JuliaLang#42766 from JuliaLang/avi/42754 optimizer: fix JuliaLang#42754, inline union-split const-prop'ed sources commit 21ebabf Author: Kristoffer Carlsson <kcarlsson89@gmail.com> Date: Tue Oct 26 16:11:32 2021 +0200 simplify code loading test now that TOML files are parsed with a real TOML parser (JuliaLang#42328) commit 1510eaa Author: Shuhei Kadowaki <aviatesk@gmail.com> Date: Mon Oct 25 01:35:12 2021 +0900 optimizer: fix JuliaLang#42754, inline union-split const-prop'ed sources This commit complements JuliaLang#39754 and JuliaLang#39305: implements a logic to use constant-prop'ed results for inlining at union-split callsite. Currently it works only for cases when constant-prop' succeeded for all (union-split) signatures. > example ```julia julia> mutable struct X # NOTE in order to confuse `fieldtype_tfunc`, we need to have at least two fields with different types a::Union{Nothing, Int} b::Symbol end; julia> code_typed((X, Union{Nothing,Int})) do x, a # this `setproperty` call would be union-split and constant-prop will happen for # each signature: inlining would fail if we don't use constant-prop'ed source # since the approximated inlining cost of `convert(fieldtype(X, sym), a)` would # end up very high if we don't propagate `sym::Const(:a)` x.a = a x end |> only |> first ``` > before this commit ```julia CodeInfo( 1 ─ %1 = Base.setproperty!::typeof(setproperty!) │ %2 = (isa)(a, Nothing)::Bool └── goto #3 if not %2 2 ─ %4 = π (a, Nothing) │ invoke %1(_2::X, 🅰️:Symbol, %4::Nothing)::Any └── goto #6 3 ─ %7 = (isa)(a, Int64)::Bool └── goto #5 if not %7 4 ─ %9 = π (a, Int64) │ invoke %1(_2::X, 🅰️:Symbol, %9::Int64)::Any └── goto #6 5 ─ Core.throw(ErrorException("fatal error in type inference (type bound)"))::Union{} └── unreachable 6 ┄ return x ) ``` > after this commit ```julia CodeInfo( 1 ─ %1 = (isa)(a, Nothing)::Bool └── goto #3 if not %1 2 ─ Base.setfield!(x, :a, nothing)::Nothing └── goto #6 3 ─ %5 = (isa)(a, Int64)::Bool └── goto #5 if not %5 4 ─ %7 = π (a, Int64) │ Base.setfield!(x, :a, %7)::Int64 └── goto #6 5 ─ Core.throw(ErrorException("fatal error in type inference (type bound)"))::Union{} └── unreachable 6 ┄ return x ) ``` commit 4c3ae20 Author: Chris Foster <chris42f@gmail.com> Date: Tue Oct 26 21:48:32 2021 +1000 Make Base.ifelse a generic function (JuliaLang#37343) Allow user code to directly extend `Base.ifelse` rather than needing a special package for it. commit 2e388e3 Author: Shuhei Kadowaki <aviatesk@gmail.com> Date: Mon Oct 25 01:30:09 2021 +0900 optimizer: eliminate excessive specialization in inlining code This commit includes several code quality improvements in inlining code: - eliminate excessive specializations around: * `item::Pair{Any, Any}` constructions * iterations on `Vector{Pair{Any, Any}}` - replace `Pair{Any, Any}` with new, more explicit data type `InliningCase` - remove dead code

This prevents us from seeing an invalid `IOStream` object from a saved system image, and also ensures the files are opened once for all threads.

JeffBezanson added randomness Random number generation and the Random stdlib bugfix This change fixes an existing bug backport 1.6 Change should be backported to release-1.6 backport 1.7 labels Oct 7, 2021

JeffBezanson force-pushed the jb/initrandomdevice branch from 0599496 to 8fdf1aa Compare October 7, 2021 22:12

KristofferC reviewed Oct 9, 2021

View reviewed changes

KristofferC mentioned this pull request Oct 11, 2021

release-1.7: Backports for 1.7-rc2 #42255

Merged

84 tasks

KristofferC mentioned this pull request Oct 19, 2021

release-1.6: Backports for julia-1.6.4 #42147

Merged

95 tasks

JeffBezanson force-pushed the jb/initrandomdevice branch from 8fdf1aa to 7f99781 Compare October 20, 2021 21:17

JeffBezanson changed the title ~~cache RandomDevice file from __init__~~ reset RandomDevice file from __init__ Oct 20, 2021

KristofferC mentioned this pull request Oct 22, 2021

release-1.7: Backports for 1.7.0/1.7.0-rc3 #42765

Merged

66 tasks

reset RandomDevice file from __init__

98f538d

This prevents us from seeing an invalid `IOStream` object from a saved system image, and also ensures the files are opened once for all threads.

KristofferC force-pushed the jb/initrandomdevice branch from 7f99781 to 98f538d Compare October 22, 2021 18:48

JeffBezanson removed the backport 1.6 Change should be backported to release-1.6 label Oct 27, 2021

JeffBezanson merged commit c74814e into master Oct 27, 2021

JeffBezanson deleted the jb/initrandomdevice branch October 27, 2021 20:34

aviatesk pushed a commit that referenced this pull request Oct 28, 2021

reset RandomDevice file from __init__ (#42537)

9528183

This prevents us from seeing an invalid `IOStream` object from a saved system image, and also ensures the files are opened once for all threads.

KristofferC removed the backport 1.7 label Nov 15, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

reset `RandomDevice` file from `init` #42537

reset `RandomDevice` file from `init` #42537

JeffBezanson commented Oct 7, 2021

KristofferC Oct 9, 2021

JeffBezanson Oct 12, 2021

KristofferC Oct 12, 2021

vtjnash Oct 12, 2021

JeffBezanson Oct 18, 2021

KristofferC commented Oct 22, 2021

reset RandomDevice file from __init__ #42537

reset RandomDevice file from __init__ #42537

Conversation

JeffBezanson commented Oct 7, 2021

KristofferC Oct 9, 2021

Choose a reason for hiding this comment

JeffBezanson Oct 12, 2021

Choose a reason for hiding this comment

KristofferC Oct 12, 2021

Choose a reason for hiding this comment

vtjnash Oct 12, 2021

Choose a reason for hiding this comment

JeffBezanson Oct 18, 2021

Choose a reason for hiding this comment

KristofferC commented Oct 22, 2021

reset `RandomDevice` file from `init` #42537

reset `RandomDevice` file from `init` #42537