Expend syntax::with_globals coverage and allow nested calls

[earthengine]

#53448 (and proberly #53469) is partialy fixed in the sense the error messages, if any, will be shown as

error: internal compiler error: broken MIR in DefId(0/0:7 ~ hrtb_fn_parameters[317d]::main[0]) (_7 = &mut (*_2)): bad assignment (&mut dyn std::ops::FnMut(()) = &mut dyn std::ops::FnMut(<() as Lt<'_>>::T)): NoSolution
  --> F:\work\rust\rust_git\rust\src/test\run-pass\hrtb-fn-parameters.rs:23:5
   |
23 |     f(v);
   |     ^

rather than "cannot access a scoped thread local variable without calling set first".

Problem analysis

run_compiler function is wrapped by a syntax::with_globals call to ensure it have access to syntax::GLOBALS. However, as we have code after running run_compile requires to emit errors, when it needs to show the position of the error, it will panic as syntax::GLOBALS is unavailable.

This pull request fixed this by introducing a new call to syntax::with_globals in run. So code in run will use bigger syntax::with_globals coverage than other calls to run_compiler.

It also makes calling syntax::with_globals when it is already called in the outside code uses the already initialized globals, rather than creating new instances.

Although this does not actually fix #53448 nor #53469, it will make similar errors never hide the useful error message any more, and so will help diagnosis simular situations easier.

---

My very first pull request on the compiler ever. Please advise me if I did anything wrong.

[rust-highfive]

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @petrochenkov (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

[petrochenkov]

r? @Zoxc

[earthengine]

So I have to stop here. The test "super-fast-paren-parsing.rs" was failed and I have no idea. As the comment was talked about "big stack" maybe an issue about a slightly bigger call stack. Need to check in a real computer.

[alexreg]

How are you getting on with this, @earthengine? Let me know if you could use some help, as I'm quite keen to get this issue solved too. Once you're done, maybe r? @oli-obk or @eddyb.

[earthengine]

It is now passed the tests. So it should be fine now!

The problem seems to be that, when the compiler needs to spawn a new thread, as I used the unwrapped call for run, it will not be covered by the with_globals call. So I reverted the refactoring (because we don't need it any more; we always assume the wrapped run_compiler), and then the updated with_globals will ensure we don't overriding the old values in the main thread.

r? @oli-obk

[oli-obk]

Can you add the ICEing test from the issue as a regression test and squash the commits?

Other than that it looks fine to me, but it seems very band-aidy. Not sure who we can ping on this though.

[Zoxc]

This fix seems questionable to me, but I haven't looked at the issues yet.

[earthengine]

@oli-obk

As this is the very first pull request, I don't know what a "regression test" mean and how to do it properly.

I was thinking adding tests though; but any tests given will be fixed at some stage though, as the code is in a error detection/prevention branch, tests for the ICEs will be fixed and so the test will be no longer appropriate.

I can squash the commits though. Will be done tonight.

[alexreg]

@earthengine I think @oli-obk just means to add your test case from the issue (a minimal reproducible version) to src/tests/run-pass/ to make sure it builds fine now, that's all. You can see other files in that directory for examples.

[earthengine]

@alexreg

The point is that ICE was NOT fixed, yet (so the minimal reproducible cannot be put in run-pass, it have to be in compile-fail to pass the test, for now). This PR can only reduce it to another ICE with more specific error message and point to a proper code position. Tests like this will not last long, not after the ICE itself being fixed.

[alexreg]

@earthengine Okay. In that case wait until you (or whoever) fixes the ICE before adding the run-pass regression test. (Do you have any idea how to fix the ICE, for that matter?)

[earthengine]

The ICE is not in my scope right now. The scope of this PR is to give more precise detail about some panicking ICEs, and so it is not targeting any specific ICE.

For this reason, I don't think we should wait for any ICEs to be fixed, as it is helping addressing issues, not fixes them.

If we DO need a test, it will be like add a compiler option -Zforce-delay-span-bug and force a delay_span_bug to trigger similar ICEs. This will require a seperate PR for that option, and wait for it to be accepted, then write a test to cause the "cannot access a scoped thread local variable without calling set first" ICE, then we can see this PR actually fixes this.

(I don't even know when I can dig deep enough to address the actual ICEs, though)

[Zoxc]

I think the clean fix here is to change fn run to call with_globals instead of fn run_compiler. The uses of run_compiler outside a run closures would then need to do their own with_globals setup.

We might be able to test ICE output to check that we get the "correct" ICE with the ui tests.

[earthengine]

I think the clean fix here is to change fn run to call with_globals instead of fn run_compiler.

This is what I have tried but didn't work. The reason is that run_compiler will be run on other threads in some configurations (for example, the "super-fast-paren-parsing.rs" test requires a bigger stack size, so the compiler spawn a new thread to run run_compiler). In that case, with_globals will not be called in that thread, and so we still have trouble.

We might be able to test ICE output to check that we get the "correct" ICE with the ui tests.

Actually I did this in my own computer! It is just that I am not sure it is good to have that test in the code base, as they will not last long.

[Zoxc]

run is the function which may create a thread (in the monitor call), and we must call with_globals after that.

[earthengine]

This is what I thought. But then with_globals will be called on a new set of empty values, I am not sure this is good as this means in most cases the previous global settings will be reset.

Even the latest form of this PR didn't resolve this though; when the new thread being created, it will work on an irrelevant instance of the globals, and so the main thread will still only get the empty values. But it only happen when there is a need to spawn a new thread and so pretty limited.

So it seems to be that the proper fix, is find a way to transfer a initialized value to the with_globals call and return it after run_compiler? This will require making sure Globals is Send

[earthengine]

My commits is now squashed without modification.

[oli-obk]

I'm going to close this in favour of #53624 since it doesn't modify the globals themselves.

Thanks for pushing ahead on this issue!