Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix tr for Symmetric/Hermitian block matrices #55522

Merged
merged 1 commit into from
Aug 19, 2024
Merged

Conversation

jishnub
Copy link
Contributor

@jishnub jishnub commented Aug 18, 2024

Since Symmetric and Hermitian symmetrize the diagonal elements of the parent, we can't forward tr to the parent unless it is already symmetric. This limits the existing tr methods to matrices of Numbers, which is the common use-case. tr for Symmetric block matrices would now use the fallback implementation that explicitly computes the diag.
This resolves the following discrepancy:

julia> S = Symmetric(fill([1 2; 3 4], 3, 3))
3×3 Symmetric{AbstractMatrix, Matrix{Matrix{Int64}}}:
 [1 2; 2 4]  [1 2; 3 4]  [1 2; 3 4]
 [1 3; 2 4]  [1 2; 2 4]  [1 2; 3 4]
 [1 3; 2 4]  [1 3; 2 4]  [1 2; 2 4]

julia> tr(S)
2×2 Matrix{Int64}:
 3   6
 9  12

julia> sum(diag(S))
2×2 Symmetric{Int64, Matrix{Int64}}:
 3   6
 6  12

@jishnub jishnub added linear algebra Linear algebra bugfix This change fixes an existing bug backport 1.10 Change should be backported to the 1.10 release backport 1.11 Change should be backported to release-1.11 labels Aug 18, 2024
@KristofferC KristofferC mentioned this pull request Aug 19, 2024
68 tasks
@jishnub jishnub merged commit 9738bc7 into master Aug 19, 2024
11 checks passed
@jishnub jishnub deleted the jishnub/sym_tr branch August 19, 2024 13:59
KristofferC added a commit that referenced this pull request Aug 26, 2024
Backported PRs:
- [x] #54962 <!-- Add timing to precompile trace compile -->
- [x] #55180 <!-- compress jit debuginfo for easy memory savings -->
- [x] #54919 <!-- Fix annotated join with non-concrete eltype iters -->
- [x] #55013 <!-- [docs] change docstring to match code -->
- [x] #55017 <!-- TOML: Make `Dates` a type parameter -->
- [x] #54033 <!-- Fix a bug in `stack`'s DimensionMismatch error message
-->
- [x] #55242 <!-- fix at-main docstring to not code quote a compat box
-->
- [x] #55261 <!-- Make `jl_*affinity` tests more portable -->
- [x] #54736 <!-- specificity: ensure fast-path in `sub/eq_msp` handle
missing `UnionAll` wrapper correctly. -->
- [x] #55299 <!-- typeintersect: fix bounds merging during inner
`intersect_all`. -->
- [x] #55302 <!-- Add `lbt_forwarded_funcs()` to debug LBT forwarding
issues -->
- [x] #55148 <!-- Random: Mark unexported public symbols as public -->
- [x] #55303 <!-- avoid overflowing show for OffsetArrays around typemax
-->
- [x] #55317 <!-- Restrict argument to `isleapyear(::Integer)` -->
- [x] #55327 <!-- Profile: Fix stdlib paths -->
- [x] #55330 <!-- [libblastrampoline] Bump to v5.11.0 -->
- [x] #55310 <!-- Preserve structure in scaling triangular matrices by
NaN -->
- [x] #55329 <!-- mapreduce: don't inbounds unknown functions -->
- [x] #55356 <!-- Profile: close files when assembling heap snapshot -->
- [x] #55371 <!-- Fix tr for block SymTridiagonal -->
- [x] #55307 <!-- Make REPL.TerminalMenus public -->
- [x] #55362 <!-- inference: fix missing LimitedAccuracy markers -->
- [x] #55306 <!-- AllocOpt: Fix stack lowering where alloca continas
boxed and unboxed data -->
- [x] #55395 <!-- fix #55389: type-unstable `join` -->
- [x] #55226 <!-- re-add `unsafe_convert` for Reinterpret and Reshaped
array -->
- [x] #55405 <!-- handle unbound vars in NTuple fields -->
- [x] #55365 <!-- ml-matches: ensure all methods are included -->
- [x] #55428 <!-- codegen: move undef freeze before promotion point -->
- [x] #55419 <!-- `stale_cachefile`: handle if the expected cache file
is missing -->
- [x] #55470 <!-- Add push! implementation for AbstractArray depending
only on resize! -->
- [x] #55483 <!-- fix hierarchy level of "API reference" in `Dates`
documentation -->
- [x] #55268 <!-- simplify complex atanh and remove singularity
perturbation -->
- [x] #55441 <!-- fix Event to use normal Condition variable -->
- [x] #55413 <!-- subtyping: fast path for lhs union and rhs typevar -->
- [x] #55492 <!-- build: add missing dependencies for expmap -->
- [x] #55507 <!-- Fix fast getptls ccall lowering. -->
- [x] #55424 <!-- add missing clamp function for IOBuffer -->
- [x] #55504 <!-- Update symmetric docstring to reflect the type of uplo
-->
- [x] #55107 <!-- Make the memory GEP an inbounds GEP since the bounds
check has happened somewhere else -->
- [x] #55411 <!-- Vendor the terminfo database for use with
base/terminfo.jl -->
- [x] #55452 <!-- Do not load `ScopedValues` with `using` -->
- [x] #55407 <!-- Remove deprecated non string API for LLVM pass
pipeline and parse all options -->
- [x] #55461 <!-- 🤖 [master] Bump the StyledStrings stdlib from d7496d2
to f6035eb -->
- [x] #55433 <!-- Backport #55407
to 1.11 -->
- [x] #55225 <!-- [1.11 backport] trace-compile: don't generate
`precompile` statements for OpaqueClosure methods (#55072) -->
- [x] #55212 <!-- Make `Base.depwarn()` public -->
- [x] #552
- [x] #55052 <!-- Fix `(l/r)mul!` with `Diagonal`/`Bidiagonal` -->
- [x] #55251 <!-- Restrict binary ops for Diagonal and Symmetric to
Number eltypes -->95 <!-- LAPACK: Aggressive constprop to concretely
infer syev!/syevd! -->
- [x] #55522 <!-- Fix tr for Symmetric/Hermitian block matrices -->

Need manual backport:
- [x] #55342 <!-- Ensure bidiagonal setindex! does not read indices in
error message -->

Contains multiple commits, manual intervention needed:

- [ ] #55336 <!-- codegen: take gc roots (and alloca alignment) more
seriously -->


Non-merged PRs with backport label:
- [ ] #55506 <!-- Fix indexing in _mapreducedim for OffsetArrays -->
- [ ] #55500 <!-- make jl_thread_suspend_and_get_state safe -->
- [ ] #55499 <!-- propagate the terminal's `displaysize` to the
`IOContext` used by the REPL -->
- [ ] #55458 <!-- Allow for generically extracting unannotated string
-->
- [ ] #55457 <!-- Make AnnotateChar equality consider annotations -->
- [ ] #55453 <!-- Privatise the annotations API, for StyledStrings -->
- [ ] #55443 <!-- Add test for upper/lower/titlecase and fix call -->
- [ ] #55355 <!-- relocation: account for trailing path separator in
depot paths -->
- [ ] #55220 <!-- `isfile_casesensitive` fixes on Windows -->
- [ ] #55169 <!-- `propertynames` for SVD respects private argument -->
- [ ] #54457 <!-- Make `String(::Memory)` copy -->
- [ ] #53957 <!-- tweak how filtering is done for what packages should
be precompiled -->
- [ ] #51479 <!-- prevent code loading from lookin in the versioned
environment when building Julia -->
- [ ] #50813 <!-- More doctests for Sockets and capitalization fix -->
- [ ] #50157 <!-- improve docs for `@inbounds` and
`Base.@propagate_inbounds` -->
- [ ] #41244 <!-- Fix shell `cd` error when working dir has been deleted
-->
@KristofferC KristofferC mentioned this pull request Aug 26, 2024
33 tasks
KristofferC pushed a commit that referenced this pull request Sep 12, 2024
Since `Symmetric` and `Hermitian` symmetrize the diagonal elements of
the parent, we can't forward `tr` to the parent unless it is already
symmetric. This limits the existing `tr` methods to matrices of
`Number`s, which is the common use-case. `tr` for `Symmetric` block
matrices would now use the fallback implementation that explicitly
computes the `diag`.
This resolves the following discrepancy:
```julia
julia> S = Symmetric(fill([1 2; 3 4], 3, 3))
3×3 Symmetric{AbstractMatrix, Matrix{Matrix{Int64}}}:
 [1 2; 2 4]  [1 2; 3 4]  [1 2; 3 4]
 [1 3; 2 4]  [1 2; 2 4]  [1 2; 3 4]
 [1 3; 2 4]  [1 3; 2 4]  [1 2; 2 4]

julia> tr(S)
2×2 Matrix{Int64}:
 3   6
 9  12

julia> sum(diag(S))
2×2 Symmetric{Int64, Matrix{Int64}}:
 3   6
 6  12
```
@KristofferC KristofferC mentioned this pull request Sep 12, 2024
63 tasks
@jishnub jishnub removed the backport 1.11 Change should be backported to release-1.11 label Sep 12, 2024
KristofferC added a commit that referenced this pull request Sep 17, 2024
Backported PRs:
- [x] #55480 <!-- Fix push! for OffsetVectors, add tests for push! and
append! on AbstractVector -->
- [x] #55443 <!-- Add test for upper/lower/titlecase and fix call -->
- [x] #55524 <!-- Set `.jl` sources as read-only during installation -->
- [x] #55500 <!-- make jl_thread_suspend_and_get_state safe -->
- [x] #55506 <!-- Fix indexing in _mapreducedim for OffsetArrays -->
- [x] #55564 <!-- Empty out loaded_precompiles dict instead of asserting
it's empty. -->
- [x] #55567 <!-- Initialize threadpools correctly during sysimg build
-->
- [x] #55596 <!-- Fast bounds-check for CartesianIndex ranges -->
- [x] #55605 <!-- Reroute Symmetric/Hermitian + Diagonal through
triangular -->
- [x] #55640 <!-- win: move stack_overflow_warning to the backtrace
fiber -->
- [x] #55715 <!-- Add precompile signatures to Markdown to reduce
latency. -->
- [x] #55593 <!-- Fix invalidations for FileIO -->
- [x] #55555 <!-- Revert "Don't expose guard pages to malloc_stack API
consumers" -->
- [x] #55720 <!-- Fix `pkgdir` for extensions -->
- [x] #55729 <!-- Avoid confounding compilation side effects of
`@time_imports` -->
- [x] #55718 <!-- Fix `@time_imports` extension recognition -->
- [x] #55522 <!-- Fix tr for Symmetric/Hermitian block matrices -->

Contains multiple commits, manual intervention needed:
- [ ] #55509 <!-- Fix cong implementation to be properly random and not
just cycling. -->

Non-merged PRs with backport label:
- [ ] #55641 <!-- fall back to slower stat filesize if optimized
filesize fails -->
- [ ] #55534 <!-- Set stdlib sources as read-only during installation
-->
- [ ] #55499 <!-- propagate the terminal's `displaysize` to the
`IOContext` used by the REPL -->
- [ ] #55458 <!-- Allow for generically extracting unannotated string
-->
- [ ] #55457 <!-- Make AnnotateChar equality consider annotations -->
- [ ] #55453 <!-- Privatise the annotations API, for StyledStrings -->
- [ ] #55355 <!-- relocation: account for trailing path separator in
depot paths -->
- [ ] #55220 <!-- `isfile_casesensitive` fixes on Windows -->
- [ ] #55169 <!-- `propertynames` for SVD respects private argument -->
- [ ] #54457 <!-- Make `String(::Memory)` copy -->
- [ ] #53957 <!-- tweak how filtering is done for what packages should
be precompiled -->
- [ ] #51479 <!-- prevent code loading from lookin in the versioned
environment when building Julia -->
- [ ] #50813 <!-- More doctests for Sockets and capitalization fix -->
- [ ] #50157 <!-- improve docs for `@inbounds` and
`Base.@propagate_inbounds` -->
- [ ] #41244 <!-- Fix shell `cd` error when working dir has been deleted
-->
KristofferC pushed a commit that referenced this pull request Oct 9, 2024
Since `Symmetric` and `Hermitian` symmetrize the diagonal elements of
the parent, we can't forward `tr` to the parent unless it is already
symmetric. This limits the existing `tr` methods to matrices of
`Number`s, which is the common use-case. `tr` for `Symmetric` block
matrices would now use the fallback implementation that explicitly
computes the `diag`.
This resolves the following discrepancy:
```julia
julia> S = Symmetric(fill([1 2; 3 4], 3, 3))
3×3 Symmetric{AbstractMatrix, Matrix{Matrix{Int64}}}:
 [1 2; 2 4]  [1 2; 3 4]  [1 2; 3 4]
 [1 3; 2 4]  [1 2; 2 4]  [1 2; 3 4]
 [1 3; 2 4]  [1 3; 2 4]  [1 2; 2 4]

julia> tr(S)
2×2 Matrix{Int64}:
 3   6
 9  12

julia> sum(diag(S))
2×2 Symmetric{Int64, Matrix{Int64}}:
 3   6
 6  12
```

(cherry picked from commit 9738bc7)
KristofferC added a commit that referenced this pull request Oct 22, 2024
Backported PRs:
- [x] #51755 <!-- ASAN fixes. -->
- [x] #55329 <!-- mapreduce: don't inbounds unknown functions -->
- [x] #55365 <!-- ml-matches: ensure all methods are included -->
- [x] #55483 <!-- fix hierarchy level of "API reference" in `Dates`
documentation -->
- [x] #55268 <!-- simplify complex atanh and remove singularity
perturbation -->
- [x] #55504 <!-- Update symmetric docstring to reflect the type of uplo
-->
- [x] #55524 <!-- Set `.jl` sources as read-only during installation -->
- [x] #41244 <!-- Fix shell `cd` error when working dir has been deleted
-->
- [x] #55829 <!-- [Dates] Make test more robust against non-UTC
timezones -->
- [x] #55641 <!-- fall back to slower stat filesize if optimized
filesize fails -->
- [x] #55849 <!-- Mmap: fix grow! for non file IOs -->
- [x] #55945 <!-- Fix logic in `?` docstring example -->
- [x] #55743 <!-- doc: heap snapshot viewing -->
- [x] #56023 <!-- Sockets: Warn when local network access not granted.
-->
- [x] #54276 <!-- Fix solve for complex `Hermitian` with non-vanishing
imaginary part on diagonal -->
- [x] #54669 <!-- Improve error message in inplace transpose -->
- [x] #55295 <!-- LAPACK: Aggressive constprop to concretely infer
syev!/syevd! -->
- [x] #55303 <!-- avoid overflowing show for OffsetArrays around typemax
-->
- [x] #55342 <!-- Ensure bidiagonal setindex! does not read indices in
error message -->
- [x] #55507 <!-- Fix fast getptls ccall lowering. -->
- [x] #55522 <!-- Fix tr for Symmetric/Hermitian block matrices -->
- [x] #55854 <!-- 🤖 [master] Bump the Downloads stdlib from 1061ecc to
89d3c7d -->
- [x] #55863 <!-- Update TaskLocalRNG docstring according to #49110 -->
- [x] #55567 <!-- Initialize threadpools correctly during sysimg build
-->
- [x] #55506 <!-- Fix indexing in _mapreducedim for OffsetArrays -->
- [x] #54737 <!-- LazyString in interpolated error messages involving
types -->
@KristofferC KristofferC removed the backport 1.10 Change should be backported to the 1.10 release label Oct 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bugfix This change fixes an existing bug linear algebra Linear algebra
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants