KeyError: key 0x0000000000000001 not found on Julia 1.8

[charleskawczynski]

Opening an issue for tracking. On Julia 1.8, I'm seeing an error:

ODE   0%\|                                               \|  ETA: N/A
ODE   0%\|                                               \|  ETA: N/A
ERROR: LoadError: KeyError: key 0x0000000000000001 not found
--
  | Stacktrace:
  | [1] getindex
  | @ ./dict.jl:498 [inlined]
  | [2] ChromeProfileFormat.CPUProfile(data::Nothing, period::Nothing; from_c::Bool)
  | @ ChromeProfileFormat /tc/depot/cpu/packages/ChromeProfileFormat/UAHXj/src/core.jl:116
  | [3] CPUProfile
  | @ /tc/depot/cpu/packages/ChromeProfileFormat/UAHXj/src/core.jl:56 [inlined]
  | [4] #save_cpuprofile#8
  | @ /tc/depot/cpu/packages/ChromeProfileFormat/UAHXj/src/core.jl:195 [inlined]
  | [5] save_cpuprofile (repeats 2 times)
  | @ /tc/depot/cpu/packages/ChromeProfileFormat/UAHXj/src/core.jl:192 [inlined]
  | [6] top-level scope
  | @ /tc/11742/turbulenceconvection-ci/perf/flame.jl:35

in TurbulenceConvection.jl, build here

[charleskawczynski]

I can try to get more actionable information for this when I get a chance

[charleskawczynski]

Here's a gist of the Profile.fetch() and Profile.getdict, collected from this build.

[charleskawczynski]

The getdict is very large, so it might make more sense to first load Profile.fetch and call getdict in the repl.

[davidanthoff]

@pfitzseb we are no longer using this package in the extension, right? If that is so, it probably means that this package is kind of unmaintained at this point...

[charleskawczynski]

Ah, I just realized that I may not have updated the call to Profile.Allocs.@profile for julia 1.8. I'll give that a shot.

[charleskawczynski]

@pfitzseb we are no longer using this package in the extension, right? If that is so, it probably means that this package is kind of unmaintained at this point...

That's unfortunate, this has been pretty helpful for us. Do you have suggestions for an alternative? cc @simonbyrne.

Btw, when trying with Profile.Allocs.@profile, I see

ODE   0%|                                               |  ETA: N/A
ODE   0%|                                               |  ETA: N/A
ERROR: LoadError: The profiling data buffer is not initialized. A profile has not been requested this session.
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] fetch(; include_meta::Bool, limitwarn::Bool)
   @ Profile /central/software/julia/1.8.0/share/julia/stdlib/v1.8/Profile/src/Profile.jl:627
 [3] fetch
   @ /central/software/julia/1.8.0/share/julia/stdlib/v1.8/Profile/src/Profile.jl:624 [inlined]
 [4] ChromeProfileFormat.CPUProfile(data::Nothing, period::Nothing; from_c::Bool)
   @ ChromeProfileFormat /tc/depot/cpu/packages/ChromeProfileFormat/UAHXj/src/core.jl:59
 [5] CPUProfile
   @ /tc/depot/cpu/packages/ChromeProfileFormat/UAHXj/src/core.jl:56 [inlined]
 [6] #save_cpuprofile#8
   @ /tc/depot/cpu/packages/ChromeProfileFormat/UAHXj/src/core.jl:195 [inlined]
 [7] save_cpuprofile (repeats 2 times)
   @ /tc/depot/cpu/packages/ChromeProfileFormat/UAHXj/src/core.jl:192 [inlined]
 [8] top-level scope
   @ /tc/11785/turbulenceconvection-ci/perf/flame.jl:35
in expression starting at /tc/11785/turbulenceconvection-ci/perf/flame.jl:29

[simonbyrne]

@davidanthoff what are you using to extract Julia profiles?

[davidanthoff]

@pfitzseb really needs to weigh in on this. All I know is that he revamped our profiling stuff in the extension completely (https://youtu.be/Okn_HKihWn8?t=763) and as far as I know the new stuff no longer uses this package. But I might also be wrong :)

[pfitzseb]

You might be able to get Valentin to look at this, since he also wrote the initial implementation :) It shouldn't be too tricky, but Profile's internals have of course changed a bit in 1.8.

Is this just for visualization? If so, aren't PProf/ProfileView/ProfileCanvas good enough?

[charleskawczynski]

Is this just for visualization

The only thing that we actually use / need is ChromeProfileFormat.save_cpuprofile

[pfitzseb]

Sure, but that's basically all this package does anyways :)

[fredrikekre]

See JuliaLang/julia#44818 (and in particular JuliaLang/julia#44818 (comment)).

[charleskawczynski]

That seems to fix it. I understand that this doesn't leverage new information, but #20 at least preserves functionality for 1.8, so we'd really appreciate support with this.

Is this just for visualization? If so, aren't PProf/ProfileView/ProfileCanvas good enough?

We're using ChromeProfileFormat to save the profile to file and upload to a cloud bucket to visualize in https://profiler.firefox.com/. AFAIK, PProf and ProfileView only provide interactive support. I haven't looked at ProfileCanvas yet, so perhaps this is an alternative, I'll take a look, thanks!

Either way, for convenience / continuity, could we merge #20?

[pfitzseb]

Both ProfileCanvas and StatProfilerHTML allow exporting static HTML, so that might be an option for you (do open issues if you find missing features/functionality or bugs in ProfileCanvas!).