Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backports for 1.9.0-rc1 #48733

Merged
merged 53 commits into from
Mar 6, 2023
Merged

Backports for 1.9.0-rc1 #48733

merged 53 commits into from
Mar 6, 2023

Conversation

KristofferC
Copy link
Member

@KristofferC KristofferC commented Feb 20, 2023

Backported PRs:

Non-merged PRs with backport label:

staticfloat and others added 6 commits February 20, 2023 12:55
Some users tried to pass a `dlopen()` handle into `lbt_forward()` which sadly works just fine, despite `ccall()` declaring its first argument as being a `Cstring`.  I guess it's trivial to convert a `Ptr{Cvoid}` into a `Cstring`, so this just goes through.

To protect against this, restrict `path` to be an `AbstractString`.

(cherry picked from commit 8492031)
…acters in path on Windows (#45126) (#45127)

* Fix jl_get_libllvm_impl to support non-ASCII characters

* Fix jl_get_libllvm_impl to support non-ASCII characters, fix whitespace

* Fix jl_get_libllvm_impl to support non-ASCII characters, fix null and buffer

(cherry picked from commit 6976bac)
@KristofferC KristofferC added the release Release management and versioning. label Feb 20, 2023
@KristofferC KristofferC force-pushed the backports-release-1.9 branch 4 times, most recently from 7a8b438 to aa1d668 Compare February 21, 2023 12:03
I found some lattice issues when implementing `MustAlias` under the new
extendable lattice system in another PR.

(cherry picked from commit ee0f3fc)
Keno and others added 14 commits February 21, 2023 17:16
We used to have this hack before #48246, but I removed it because I had
hoped we don't need. Unfortunately, we weren't able to infer consistency of
```
@inbounds (1,2)[1]
```
With this hack, constprop is able to independently prove inbounded-ness,
overriding the usual consistency taintaing that happens for inbounds.

(cherry picked from commit 113c2f3)
Remove the high watermark logic, because it doesn't really make sense,
and allow for use of 60% of system memory before aggressive GC kicks in.

Should fix #48473

(cherry picked from commit 500f561)
Many of these test were hiding output or using semi-deprecated functions
or failing to cleanup output or setting the wrong variables.

(cherry picked from commit b48104d)
* effects: taint `:nothrow` effect on unknown `:static_parameter` (conservatively)

With this commit, we taint `:nothrow` effect property correctly on
access to unknown `:static_parameter`, e.g.:
```julia
unknown_sparam_throw(::Union{Nothing, Type{T}}) where T = (T; nothing)
@test Core.Compiler.is_nothrow(Base.infer_effects(unknown_sparam_throw, ((Type{Int},))))
@test !Core.Compiler.is_nothrow(Base.infer_effects(unknown_sparam_throw, ((Nothing,))))
```

This commit implements a very conservative analysis, and thus there is a
room for improvement still, e.g.:
```julia
unknown_sparam_nothrow(x::Ref{T}) where {T} = (T; nothing)
@test_broken Core.Compiler.is_nothrow(Base.infer_effects(unknown_sparam_nothrow, (Ref,)))
```

* inference: improve `:nothrow` modeling for `:static_parameter` (#46820)

* Fix test with free type params

* Test: Ignore ::Type{T} in detect_unbounded

These are only technically unbounded because of the existence of
ill-formed types. However, this function is supposed to be an API
sanity check and ordinary users should never have ill-formed types,
so for the purpose we want here, allow unboundedness in Type{T}.

---------

Co-authored-by: Keno Fischer <keno@juliacomputing.com>
Co-authored-by: Elliot Saba <staticfloat@gmail.com>
(cherry picked from commit b5d17ea)
* Run dsymutil on pkgimgs for apple platforms +
LLD bump to get dsymutil from the jll

(cherry picked from commit d8c2250)
* [OpenBLAS_jll] Upgrade to new build optimised for PowerPC

This only difference compared to previous build is that this one enables use of
dynamic architecture detection also for the PowerPC architecture.

---------

Co-authored-by: Valentin Churavy <v.churavy@gmail.com>
(cherry picked from commit aacfcf0)
On 64-bit, we have enough space to encode (1) the tag, (2) the
`depmods` index, and (3) the offset all in a single 64-bit pointer
field. This means we don't need the external `link_id` arrays,
which reduces the size of many pkgimages by ~5%.

On 32-bit, we don't have enough bits to implement this strategy.
However, most linkages seem to be against the sysimage, and so
by giving that a separate tag we can achieve similar compression
because the `link_id` lists will be much shorter.

Co-authored-by: Tim Holy <tim.holy@gmail.com>

(cherry picked from commit 8e3e970)
KristofferC and others added 10 commits March 3, 2023 11:29
We need to make sure that libjulia-internal is loaded and placed first in the
DynamicLibrary search order so that calls to runtime intrinsics are resolved
to the correct library when multiple libjulia-*'s have been loaded (e.g. when
we `ccall` into a PackageCompiler.jl-created shared library).

(cherry picked from commit fd45943)
* Update MbedTLS to 2.28.2

* Update deps/mbedtls.mk

* Update mbedtls checksum

(cherry picked from commit 127fb73)
* Always use `-Wl,--warn-unresolved-symbols` in MSAN builds

* Use `-Wl,--warn-unresolved-symbols` only on Linux and FreeBSD

(cherry picked from commit a07f512)
Co-authored-by: Jameson Nash <vtjnash@gmail.com>
Co-authored-by: Dilum Aluthge <dilum@aluthge.com>
(cherry picked from commit 6412a56)
* enable using external code in pkgimages

This was unintentionally disabled (and incomplete) in the original PR
for pkgimages.

Co-authored-by: Valentin Churavy <v.churavy@gmail.com>
(cherry picked from commit c136b7e)
Memoizing this can save a lot of repeated effort for queries such as
`@which eltype(String)`. Otherwise we repeatedly try to check if
`eltype(::Type)` is a good way to cache that result, even though it
never gets better the more we check it.

(cherry picked from commit bdcd5e2)
KristofferC and others added 4 commits March 3, 2023 12:06
Previously we would double-account for many of these, leading to
occasional chaos. Try to avoid serializing code that does not belong to
this incremental compilation session package.

Refs: #48723
(cherry picked from commit 81f366d)
There is actually almost no cases where `jl_alloc_svec_uninit` is safe,
since if was safe, you would likely would prefer to use the `jl_svec`
constructor instead.

(cherry picked from commit 0608824)
@vchuravy
Copy link
Member

vchuravy commented Mar 5, 2023

vchuravy@odin ~/b/julia-1.9> make -C test misc
make: Entering directory '/home/vchuravy/builds/julia-1.9/test'
    JULIA /home/vchuravy/builds/julia-1.9/test/misc
Running parallel tests with:
  nworkers() = 1
  nthreads() = 1
  Sys.CPU_THREADS = 16
  Sys.total_memory() = 125.703 GiB
  Sys.free_memory() = 103.313 GiB

Test  (Worker) | Time (s) | GC (s) | GC % | Alloc (MB) | RSS (MB)
misc       (1) |        started at 2023-03-05T13:45:36.168
┌ Warning: `old_return_args` is deprecated, use `new_return_args` instead.
│   caller = macro expansion at Test.jl:478 [inlined]
└ @ Core ~/builds/julia-1.9/usr/share/julia/stdlib/v1.9/Test/src/Test.jl:478
┌ Warning: `old_return_args` is deprecated, use `new_return_args` instead.
│   caller = macro expansion at Test.jl:478 [inlined]
└ @ Core ~/builds/julia-1.9/usr/share/julia/stdlib/v1.9/Test/src/Test.jl:478
┌ Warning: skipping stftime tests: no locale found for testing
└ @ Main.Test13Main_misc ~/src/julia/test/misc.jl:1099
misc       (1) |         failed at 2023-03-05T16:54:28.430
Test Failed at /home/vchuravy/src/julia/test/misc.jl:496
  Expression: summarysize(Core) > summarysize(Core.Compiler) + Base.summarysize(Core.Intrinsics) > Core.sizeof(Core)
   Evaluated: 42790095 > 60977967 > 0

Test Summary: |    Pass  Fail    Total       Time
  Overall     | 1282259     1  1282260  188m53.6s
    misc      | 1282259     1  1282260  188m52.7s
    FAILURE

The global RNG seed was 0xbd82c5bcade1a3be3610e82bec31468a.

Error in testset misc:
Test Failed at /home/vchuravy/src/julia/test/misc.jl:496
  Expression: summarysize(Core) > summarysize(Core.Compiler) + Base.summarysize(Core.Intrinsics) > Core.sizeof(Core)
   Evaluated: 42790095 > 60977967 > 0

188m for this testset is exceeding our timeout on master this takes 52.53s...

src/method.c Outdated Show resolved Hide resolved
vchuravy and others added 8 commits March 5, 2023 20:47
This ensures that the value gets set after all possible compilation
has occurred.

(cherry picked from commit 0ec704e)
(cherry picked from commit 39e34ee)
* Fix sorting missing values with offset indices

* Add tests

(cherry picked from commit 7eb9615)
* Fix heapsize hint and use a line
so that large machines utilize more of their ram

(cherry picked from commit 51db8af)
@KristofferC
Copy link
Member Author

@nanosoldier runtests(ALL, vs = "@7b395153e80672f8cdb18f51dd653a85e28b2070", configuration = (buildflags=["LLVM_ASSERTIONS=1", "FORCE_ASSERTIONS=1"],), vs_configuration=(buildflags=["LLVM_ASSERTIONS=1", "FORCE_ASSERTIONS=1"],))

@nanosoldier
Copy link
Collaborator

Your package evaluation job has completed - possible new issues were detected.
A full report can be found here.

@KristofferC KristofferC merged commit 94603a7 into release-1.9 Mar 6, 2023
@KristofferC KristofferC deleted the backports-release-1.9 branch March 6, 2023 21:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release Release management and versioning.
Projects
None yet
Development

Successfully merging this pull request may close these issues.