Move debug-info and extra-debug-info out of the -Z flag

[bstrie]

Now that #9658 has landed, libstd is buildable with both -Z debug-info and -Z extra-debug-info. It might be time to graduate these flags out of the experimental -Z zone.

[bstrie]

cc @michaelwoerister

[michaelwoerister]

I'm not sure it's quite there yet. The most obvious issue is that source code positions are not yet correctly set in LLVM IR in some very visible places (like if-expressions and function calls). This will easily (and understandably) lead to confusion among users (see #9641 for an example).

Another issue is that activating debug info doesn't seem to be compatible with activating optimizations (i.e. the -O flag). It's not always a problem but especially with extra-debug-info LLVM very often runs into an assertion when finalizing the module. I'm not sure yet what this is about. Is it because I'm doing something wrong with LLVM's resource management? Is it because we are uncovering a bug in LLVM that just hasn't shown in Clang yet? Or because it's just not supported by LLVM... (any insights on this --- that is LLVM/Clang and optimizations+debug info --- would be very welcome, @jdm @brson @thestinger @alexcrichton and other LLVM veterans)

In any case, for the time being I guess we should at least print out a warning when debug info and optimization are turned on at the same time.

[alexcrichton]

I could certainly go through upgrading LLVM again, I'd just be shooting in the dark, though. I'll see if any of the commit messages look relevant. I know that @thestinger and others found this awesome tool in LLVM to narrow down bitcode files to something that we could report to LLVM, although perhaps it's just particular passes which interfere with debuginfo...

Regardless, I don't think that we should promote this to a proper flag if we still cause lots of compilation problems with optimizations on. That being said, I'm perfectly OK with a warning being printed if you requested optimizations and debug info and then debug info is just not generated.

[thestinger]

@michaelwoerister: if you're hitting an assertion, you can have rustc output a bytecode file with --emit-llvm and use bugpoint on it to cut it down to a minimal test case

[pnkfelix]

Maybe a less aggressive first step would be a configure flag that would make us attempt to build rustc with the -Z debug-info and -Z extra-debug-info switches. It certainly would be a stress test for the feature.

[jdm]

Note that extra-debug-info implies debug-info, and might even be overwritten if the latter appears later in the command line.

[michaelwoerister]

@alexcrichton Thanks for the offer but, as you say, updating LLVM is unlikely to solve the problem. Better not invest any time if it's just for this.

@thestinger Thanks for the hint. bugpoint looks very useful. However, if I remember correctly, the LLVM assertion occurs during the translation pass (in DIBuilder::finalize()) so we never get to the point where we have a valid .bc file. Or maybe I misunderstood how bugpoint/the trans workflow works?

I think the best course here is for me to spend a day or two to really dig into the assertion problem. Maybe I can allocate some time for that next week. For now I'll create an issue describing the problem.

As far as moving debug info out of the -Z flags is concerned, I still think that should wait until source code stepping is in better shape and we get no crashes with optimized builds. Or, if the optimization thing turns out to be more difficult than expected, the option with the warning about not mixing debug info and optimizations.

[michaelwoerister]

A little update on this: After 2 days of digging into it I was able to locate the issue causing the LLVM assertion about metadata still being referenced after deletion. It's a memory management issue in LLVM where some objects are never freed in rare cases. There is already a bug report on the same issue in LLVM's bug tracker and I added some more details to it in case somebody is interested.

How do we go about fixing issues in LLVM? Wait until they get fixed upstream? Fix them in our version?

[jdm]

If the issues cause pain in our work and we have solutions we're confident in, we tend to land locally and work on upstreaming, I believe.

[michaelwoerister]

Well, the fix I propose in the LLVM bug report will certainly get rid of the memory leak but I don't know enough about the inner workings of LLVM's debug info handling to know whether it would be better to solve the problem earlier. That is, instead of making the function resilient against duplicates in sub-program list, it would also make sense to dedup the list already earlier.

I can try to make sure that we don't end up with duplicates in the first place (which, I think, is related to #7349). That would solve the problem on the Rust side and we would not have to wait on LLVM to fix the issue.

[alexcrichton]

Any updates on this? Perhaps this has baked long enough to make it to graduation?

[michaelwoerister]

The issues mentioned earlier in this thread have all been solved. Also, thanks to @comex, there's also a preliminary debugging for Mac OS now. I think, I'd be OK with moving debuginfo to -g and -gline-tables-only, or whatever we want to call the command line option.

[alexcrichton]

Adding E-easy and E-mentor, I'd be willing to mentor this.