Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

put Logging back in default sysimage #53660

Closed
wants to merge 1 commit into from
Closed

Conversation

JeffBezanson
Copy link
Member

fixes #51493, fixes #52075

@JeffBezanson JeffBezanson added the backport 1.11 Change should be backported to release-1.11 label Mar 8, 2024
@KristofferC
Copy link
Member

KristofferC commented Mar 8, 2024

And now StyledStrings is in there as well? That feels like something people will really start relying on. @tecosaur, wasn't the plan to move the styled string rendering code into Base (which would allow Consologger to also be moved into Base. Alternatively the use of StyledStrings in logging could be reverted.

@JeffBezanson
Copy link
Member Author

Yes, one option is to move ConsoleLogger to base and merge it with SimpleLogger. Then it would be nice if it can avoid using StyledStrings, but I'm not sure that's possible without sacrificing functionality people want.

@IanButterworth
Copy link
Member

Could we add a class of excised stdlibs that are pkgimages that are loaded immediately after the sysimage, before startup.jl, but after the active project is set, and there could be a flag to disable it.
That way they can still be upgradable and avoidable if people want.

@tecosaur
Copy link
Contributor

tecosaur commented Mar 9, 2024

@tecosaur, wasn't the plan to move the styled string rendering code into Base

The (inconclusive) discussion on Triage partially recorded in Slack is this: https://julialang.slack.com/archives/C67TUQBG9/p1708007625965089

For posterity, I'll copy over the first (and most relevant) message here before it gets eaten by the slack-hole:

  1. We always load StyledStrings (in it's current form) as part of startup, unless you specifically build a sysimage/executable where you've opted-out. We keep the print(::IO, ::AnnotatedString) / show(::IO, MIME"text/plain", ::AnnotatedString) methods defined in StyledStrings (similar to LinearAlgebra).
    1.1: We can introduce a class of stdlibs that are loaded immediately after the stdlibs (e.g. Logging, LinearAlgebra?)
    1.2: These stdlibs can be baked into the distributed sysimage (but we now have upgradible stdlibs!)
  2. We move the core "how to print an AnnotatedString with the styling info as ANSI escape sequences" functionality into Base, the StyledStrings stdlib contains "the other stuff" (load user customisations, convenience functions).

I don't really have a horse in this race, so long as we don't go down the "let's not use StyledStrings for stuff in Base/not-REPL stdlibs" path, since I have yet more hopes for using it to enhance other aspects like error messages (if it's somehow possible, I'd love to be able to use JuliaSyntaxHighlighting with some error messages/stacktraces, see also https://julialang.slack.com/archives/C67TUQBG9/p1709307800994979).

That said, I do personally somewhat like the sound of Ian's suggestion on the face of it 🙂.

@JeffBezanson
Copy link
Member Author

I'm fine with any of that, but I want to avoid doing significant development work / adding features in order to fix a regression. For 1.11, it looks to me like we need to put Logging back where it was and re-try moving it out when we have everything sorted to make that possible.

@IanButterworth
Copy link
Member

Is this mergeable or does the StyledStrings consideration require changes?

@KristofferC KristofferC mentioned this pull request Mar 15, 2024
60 tasks
KristofferC added a commit that referenced this pull request Mar 17, 2024
Backported PRs:
- [x] #39071 <!-- Add a lazy `logrange` function and `LogRange` type -->
- [x] #51802 <!-- Allow AnnotatedStrings in log messages -->
- [x] #53369 <!-- Orthogonalize re-indexing for FastSubArrays -->
- [x] #48050 <!-- improve `--heap-size-hint` arg handling -->
- [x] #53482 <!-- add IR encoding for EnterNode -->
- [x] #53499 <!-- Avoid compiler warning about redefining jl_globalref_t
-->
- [x] #53507 <!-- update staled `Core.Compiler.Effects` documentation
-->
- [x] #53408 <!-- task splitting: change additive accumulation to
multiplicative -->
- [x] #53523 <!-- add back an alias for `check_top_bit` -->
- [x] #53377 <!-- add _readdirx for returning more object info gathered
during dir scan -->
- [x] #53525 <!-- fix InteractiveUtils call in Base.runtests on failure
-->
- [x] #53540 <!-- use more efficient `_readdirx` for tab completion -->
- [x] #53545 <!-- use `_readdirx` for `walkdir` -->
- [x] #53551 <!-- revert "Add @create_log_macro for making custom styled
logging macros (#52196)" -->
- [x] #53554 <!-- Always return a value in 1-d circshift! of
abstractarray.jl -->
- [x] #53424 <!-- yet more atomics & cache-line fixes on work-stealing
queue -->
- [x] #53571 <!-- Update Documenter to v1.3 for inventory writing -->
- [x] #53403 <!-- Move parallel precompilation to Base -->
- [x] #53589 <!-- add back `unsafe_convert` to pointer for arrays -->
- [x] #53596 <!-- build: remove extra .a file -->
- [x] #53606 <!-- fix error path in `precompilepkgs` -->
- [x] #53004 <!-- Unexport with, at_with, and ScopedValue from Base -->
- [x] #53629 <!-- typo fix in scoped values docs -->
- [x] #53630 <!-- sroa: Fix incorrect scope counting -->
- [x] #53598 <!-- Use Base parallel precompilation to build stdlibs -->
- [x] #53649 <!-- precompilepkgs: package in boths deps and weakdeps are
in fact only weak -->
- [x] #53671 <!-- Fix bootstrap Base precompile in cross compile
configuration -->
- [x] #52125 <!-- Load Pkg if not already to reinstate missing package
add prompt -->
- [x] #53602 <!-- Handle zero on arrays of unions of number types and
missings -->
- [x] #53516 <!-- permit NamedTuple{<:Any, Union{}} to be created -->
- [x] #53643 <!-- Bump CSL to 1.1.1 to fix libgomp bug -->
- [x] #53679 <!-- move precompile workload back from Base -->
- [x] #53663 <!-- add isassigned methods for reinterpretarray -->
- [x] #53662 <!-- [REPL] fix incorrectly cleared line after completions
accepted -->
- [x] #53611 <!-- Linalg: matprod_dest for Diagonal and adjvec -->
- [x] #53659 <!-- fix #52025, re-allow all implicit pointer casts in
cconvert for Array -->
- [x] #53631 <!-- LAPACK: validate input parameters to throw informative
errors -->
- [x] #53628 <!-- Make some improvements to the Scoped Values
documentation. -->
- [x] #53655 <!-- Change tbaa of ptr_phi to tbaa_value  -->
- [x] #53391 <!-- Default to the medium code model in x86 linux -->
- [x] #53699 <!-- Move `isexecutable, isreadable, iswritable` to
`filesystem.jl` -->
- [x] #41232 <!-- Fix linear indexing for ReshapedArray if the parent
has offset axes -->
- [x] #53527 <!-- Enable analyzegc checks for try catch and fix found
issues -->
- [x] #52092 
- [x] #53682 <!-- Increase build precompilation -->
- [x] #53720 
- [x] #53553 <!-- typeintersect: fix `UnionAll` unaliasing bug caused by
innervars. -->

Contains multiple commits, manual intervention needed:
- [ ] #53305 <!-- Propagate inbounds in isassigned with CartesianIndex
indices -->

Non-merged PRs with backport label:
- [ ] #53736 <!-- fix literal-pow to return the right type when the base
is -1 -->
- [ ] #53707 <!-- Make ScopedValue public -->
- [ ] #53696 <!-- add invokelatest to on_done callback in bracketed
paste -->
- [ ] #53660 <!-- put Logging back in default sysimage -->
- [ ] #53509 <!-- revert moving "creating packages" from Pkg.jl -->
- [ ] #53452 <!-- RFC: allow Tuple{Union{}}, returning Union{} -->
- [ ] #53402 <!-- Add `jl_getaffinity` and `jl_setaffinity` -->
- [ ] #52694 <!-- Reinstate similar for AbstractQ for backward
compatibility -->
- [ ] #51928 <!-- Styled markdown, with a few tweaks -->
- [ ] #51816 <!-- User-themable stacktraces -->
- [ ] #51811 <!-- Make banner size depend on terminal size -->
- [ ] #51479 <!-- prevent code loading from lookin in the versioned
environment when building Julia -->
@KristofferC KristofferC mentioned this pull request Mar 20, 2024
41 tasks
@LilithHafner LilithHafner added this to the 1.11 milestone Apr 10, 2024
@LilithHafner
Copy link
Member

Adding to milestone to avoid logging UI regression.

@JeffBezanson
Copy link
Member Author

StyledStrings.__init__ relies on DEPOT_PATH being non-empty, but we have an explicit test for that case. Should the init code just check for that and do nothing?

@KristofferC
Copy link
Member

There's been talk for months now about how the rendering of StyledStrings has to be moved into Base if it wants to style things in Base and the same applies here since the default logging of Base should be "nice" but nothing happens. So I will just revert the StyledString usage in Logging, move ConsoleLogger.jl into Base so this is finally dealt with.

@tecosaur
Copy link
Contributor

StyledStrings.init relies on DEPOT_PATH being non-empty...

Thanks to Kristoffer, this is now tracked as JuliaLang/StyledStrings.jl#58

There's been talk for months now about how the rendering of StyledStrings has to be moved into Base if it wants to style things in Base and the same applies here since the default logging of Base should be "nice" but nothing happens

This is an interpretation of events that differs from mine, as I'm still under the impression that there were ~3 options put forward to consider, and I don't have strong opinions on them and so have been waiting for one to be picked.

I'm not too fussed, but if/when the new HelpfulErrors stdlib mentioned briefly in the second last triage call gets off the ground, that will be a repeat of the Logging situation, except with "just revert the use of AnnotatedStrings for styling" being less straightforward.

For a bit more context (as I recall, at least), Triage suggested that in this case that it could potentially have StyledStrings as a dep, and be loaded in interactive sessions and possibly by the compiler for nicer error messages.

@KristofferC
Copy link
Member

KristofferC commented May 13, 2024

This is an interpretation of events that differs from mine, as I'm still under the impression that there were #53660 (comment),

What I am saying is exactly point 2 there?

I'm not too fussed, but if/when the new HelpfulErrors stdlib mentioned

Triage suggested that in this case that it could potentially have StyledStrings as a dep, and be loaded in interactive sessions and possibly by the compiler for nicer error messages.

This would not be an stdlib. It would just be something that Base can do. And that would require the same as here, being able to render styled strings from Base.

@tecosaur
Copy link
Contributor

What I am saying is exactly point 2 there?

I may just be misinterpreting — I read your comment as saying that's the only way forward.

This would not be an stdlib. It would just be something that Base can do.

My "a potential new stdlib" comment is from when I popped into a recent triage meeting and basically asked "here's some functionality I think could be worth sorting on, what do you think and where should it go?" and the preliminary answer I got was "make it a new stdlib".

@KristofferC
Copy link
Member

I read your comment as saying that's the only way forward.

Yes, the other ones don't make sense.

the preliminary answer I got was "make it a new stdlib".

Well, that's just wrong... Error messages is a Base feature, it doesn't make sense for a "HelpfulErrors" stdlib. Either it would be a normal Base feature if it is stable and "good" enough, or it would be a separate package.

@KristofferC
Copy link
Member

ConsoleLogging got put into Base.

@LilithHafner
Copy link
Member

I concur with traige's position that HelpfulErrorMessages should be a stdlib, not in Base. Exceptions and try/catch should be a part of Base, basic error messages should also imo be a part of base, but fancy helpful errors that try to provide insightful advice need not be. They can and imo should pirate base methods or hook into error hint API for this. HelpfulErrorMessages should be loaded by default in the REPL and when invoking julia from the command line.

@KristofferC
Copy link
Member

try/catch should be a part of Base,

What?

They can and imo should pirate base methods

This would be terrible for latency.

hook into error hint API for this.

Yes, the existing error hint API sits there ready for contributions.

@tecosaur
Copy link
Contributor

Should we move this tangent into a dedicated issue/elsewhere?

Anyway, my takeaway was that Base itself could just define and throw specialised error types, with nice showerror methods in HelpfulErrors. This is technically type piracy, but it's invalidations that cause latency, not piracy, no?

@giordano giordano deleted the jb/sysimgLogging branch June 19, 2024 19:42
@KristofferC KristofferC removed the backport 1.11 Change should be backported to release-1.11 label Aug 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ConsoleLogger not active in the REPL Fallback Logger used during startup.jl
5 participants