fix more of the #18017 regression

[JeffBezanson]

This is a better fix that should take care of the remaining regressions.

[JeffBezanson]

@nanosoldier runbenchmarks(ALL, vs="@b80dad4")

[vtjnash]

lgtm

[nanosoldier]

Something went wrong when running your job: RemoteException(2,CapturedException(ErrorException("type NanosoldierError is immutable"),Any[( in report at BenchmarkJob.jl:405,1),( in run at BenchmarkJob.jl:233,1),( in #503 at multi.jl:1410,1),( in run_work_thunk at multi.jl:996,1),( in macro expansion at multi.jl:1410 [inlined],1),( in #502 at event.jl:46,1)]))
cc @jrevels

[JeffBezanson]

@nanosoldier runbenchmarks(ALL, vs="@b80dad4")

[nanosoldier]

Something went wrong when running your job: RemoteException(3,CapturedException(ErrorException("type NanosoldierError is immutable"),Any[( in report at BenchmarkJob.jl:405,1),( in run at BenchmarkJob.jl:233,1),( in #503 at multi.jl:1410,1),( in run_work_thunk at multi.jl:996,1),( in macro expansion at multi.jl:1410 [inlined],1),( in #502 at event.jl:46,1)]))
cc @jrevels

[KristofferC]

https://github.com/JuliaCI/BaseBenchmarkReports/blob/73b77790dbeafd8a7db4e3806cc48874c2f7e51b/260852e_vs_b80dad4/report.md

Make seemed to fail.

[tkelman]

configure: WARNING: libpsl was not found
configure: error: one or more libs available at link-time are not available run-time. Libs used at link-time: -lssh2 -lmbedtls -lmbedx509 -lmbedcrypto -lz 

Does nanosoldier do each build from a totally clean state, or does it reuse the artifacts of one when it builds the other?

[jrevels]

Builds from a clean state. The relevant function is here.

[tkelman]

So did #18103 break a clean build from master then? edit: yep.

[JeffBezanson]

@nanosoldier runbenchmarks(ALL, vs="@b80dad4")

[nanosoldier]

Something went wrong when running your job: RemoteException(3,CapturedException(ErrorException("type NanosoldierError is immutable"),Any[( in report at BenchmarkJob.jl:405,1),( in run at BenchmarkJob.jl:233,1),( in #503 at multi.jl:1410,1),( in run_work_thunk at multi.jl:996,1),( in macro expansion at multi.jl:1410 [inlined],1),( in #502 at event.jl:46,1)]))
cc @jrevels

[tkelman]

Make is failing, and will until (at least part of) #18153 is merged.

Is this ready to merge or do you want to be absolutely sure it doesn't cause regressions beforehand?

[JeffBezanson]

I did want to run the benchmarks to confirm the fix, but this should be fine to merge.

[tkelman]

daily build is working, but comment triggers aren't?
@nanosoldier runbenchmarks(ALL, vs="@b80dad4")