[Nightlies] Latest VS2017 hard crashes when opening a solution with latest nightly

[isaacabraham]

I have a solution (new style .NET Framework project) which when I try to load into VS 2017 (15.7.6), VS will hard crash during startup - no error dialogs or anything, the app simply disappears.

Here's the event log that I was able to find:

Application: devenv.exe
Framework Version: v4.0.30319
Description: The application requested process termination through System.Environment.FailFast(string message).
Message: System.NullReferenceException: Object reference not set to an instance of an object.
   at Microsoft.CodeAnalysis.Editor.Implementation.Structure.AbstractStructureTaggerProvider`1.<ProduceTagsAsync>d__6.MoveNext()
Stack:
   at System.Environment.FailFast(System.String, System.Exception)
   at Microsoft.CodeAnalysis.FailFast.OnFatalException(System.Exception)
   at Microsoft.CodeAnalysis.ErrorReporting.FatalError.Report(System.Exception, System.Action`1<System.Exception>)
   at Microsoft.CodeAnalysis.ErrorReporting.FatalError.ReportUnlessCanceled(System.Exception)
   at Microsoft.CodeAnalysis.Editor.Implementation.Structure.AbstractStructureTaggerProvider`1+<ProduceTagsAsync>d__6[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].MoveNext()
   at Microsoft.CodeAnalysis.Editor.Implementation.Structure.AbstractStructureTaggerProvider`1+<ProduceTagsAsync>d__6[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore+MoveNextRunner.InvokeMoveNext(System.Object)
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore+MoveNextRunner.Run()
   at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action, Boolean, System.Threading.Tasks.Task ByRef)
   at System.Threading.Tasks.Task.FinishContinuations()
   at System.Threading.Tasks.Task.FinishStageThree()
   at System.Threading.Tasks.Task`1[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].TrySetResult(System.__Canon)
   at System.Threading.Tasks.TaskCompletionSource`1[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].TrySetResult(System.__Canon)
   at Microsoft.VisualStudio.FSharp.Editor.RoslynHelpers+StartAsyncAsTask@111-4[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].Invoke(System.Exception)
   at Microsoft.FSharp.Control.AsyncPrimitives+StartWithContinuations@947-2.Invoke(System.Runtime.ExceptionServices.ExceptionDispatchInfo)
   at Microsoft.FSharp.Control.Trampoline.Execute(Microsoft.FSharp.Core.FSharpFunc`2<Microsoft.FSharp.Core.Unit,Microsoft.FSharp.Control.AsyncReturn>)
   at Microsoft.FSharp.Control.TrampolineHolder.ExecuteWithTrampoline(Microsoft.FSharp.Core.FSharpFunc`2<Microsoft.FSharp.Core.Unit,Microsoft.FSharp.Control.AsyncReturn>)
   at <StartupCode$FSharp-Core>.$Async+-ctor@167-1.Invoke(System.Object)
   at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context(System.Object)
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()

I have another app that seems to open without crashing, but the F# tools don't seem to start up - there's no colouration or anything.

Can't share the code publicly I'm afraid.

[isaacabraham]

I can confirm that removing the latest nightly (6 August) rolls back to something in June or July which does work.

[TIHan]

Thank you for reporting this issue.

Lately we have seen some NRE issues in mixed solutions. Are you running a mixed solution? What was your previous version of VS before you updated to 15.7.6?

Roslyn has resolved a few NRE issues in 15.8 preview5 related to F#. I'm curious if they resolve yours.

The other issue you mentioned of no colorization should be resolved in 15.8 rtm. We made a perf tweak in 15.7.3 that caused this; it slipped through us due to how specific the case was.

[TIHan]

Based on your latest comment, you mention you are using nightlies. Hmm. This makes the problem a bit more tricky.

[isaacabraham]

@TIHan I was using 15.7.5 previously with the nightlies and it was working absolutely fine. The problem might be from 15.7.6 but could be one of the newer nightlies that are causing it.

This solution is an F#-only one.

I really don't want to start using the VS previews if I can avoid it :-/

[TIHan]

Ok, thank you for the info.

You won't have to wait much longer for 15.8 rtm as release will be soon.

[cartermp]

@isaacabraham It's likely from the nightlies; we haven't had an insertion into VS 15.7 since 15.7.4. Anything in master (likely very recently) will be causing this.

[cartermp]

We'll need to get to the bottom of this just to make sure it doesn't affect any 15.9 payloads, though

[isaacabraham]

OK. What can I do to help - happy to do a screen share or something if that helps?

[TIHan]

I wouldn't be surprised if latest master is causing this in 15.7.

We will most likely need a new public project that can reproduce this.

[isaacabraham]

OK, good(ish) news - whilst the crashing is not happening on a brand new console application, I'm not getting colouration or anything else, which suggests something with my VS install. What's next?

[cartermp]

@isaacabraham Try the following:

1. Open the solution
2. Ensure an F# document is open
3. Close the F# document

If you get colors/etc, then this is a known issue that has been fixed upstream in Roslyn. If not, then we'll have to investigate further.

[isaacabraham]

@cartermp Unfortunately that did not solve the issue. I can also see now that when opening an fsx file, I see this error:

Don't know if that helps at all.

[jmarolf]

butting my head in here, there are a few causes of the error you are seeing:

0. The VSIX installer did not correctly add binding redirects to the devenv.exe.config file for the F# nightlies causing a mish-mash of assemblies to load.
1. The package for F# is authored incorrectly or there is a race in its initialization that allows it to declare itself loaded without all necessary components having actually finished loading (a more advanced version of this or perhaps this very bug manifesting in 15.7).
2. The version of roslyn that the nightlies are built against does not match what is in 15.7 (unlikely, nightlies are built from master which I assume needs to be F5 compatible with the latest public release of VS).

Personal guess is on 1. But @TIHan would know more.

To eliminate 0. uninstall and re-install the nightlies and run devenv.exe /UpdateConfiguration

[isaacabraham]

@jmarolf thanks for the idea - just tried /updateconfiguration but sadly no help. I'm thinking that since other people aren't complaining about this, there's something screwy on my machine.

[ricardozimmer]

I see a similar behavior, VS Community 2017 V 15.7.6 when installing nightly build F# Tools. Never crashed but editor doesn't get any color, and right side of the keyboard inoperative such as numpads, enter, backspace. I can type letters like aaaaaaa but no way to delete them. Uninstalling, works fine. I noticed this behavior 2 releases ago (I believe). Hope this helps.

[mBHyuC]

@isaacabraham Not alone, same for me. Option 0 does not fix it.

Anwendung: devenv.exe
Frameworkversion: v4.0.30319
...
   bei Microsoft.CodeAnalysis.Editor.Implementation.Structure.AbstractStructureTaggerProvider`1.<ProduceTagsAsync>d__6.MoveNext()

[TIHan]

Judging by everyone's comments, it looks like the F# package is not loading. @ricardozimmer is correct that it happened just a few releases for him because we introduced a perf tweak that allowed this problem to show. In 15.8 RTM we have this resolved at the Roslyn level where the F# package will be loaded correctly.

[ricardozimmer]

@TIHan Great, thanks!!

[cartermp]

Note that this is in latest master, so f5 from the tools is unusable.

[cartermp]

Given @TIHan's comment, this may not be fully resolved until 15.8 is merged into master, and people use 15.8.

This means that the F# nightlies will likely remain unusable until 15.8.

[isaacabraham]

@cartermp will installing the latest 15.8 preview fix this?

[jmarolf]

Looking at the dump I have of the crash (I was able to reproduce this locally) I can confirm that this is 1. Using the 15.8 preview or waiting for 15.8 to be released is the best solution I have now.

[cartermp]

@isaacabraham Perhaps (upstream Roslyn change is not in Preview 5 IIRC), but you can always just back off of nightlies until the release.

[cartermp]

@isaacabraham have you had a chance to upgrade to VS 15.8 to test if this still reproduces?

Just for safety's sake I'd recommend a separate machine or VM

[isaacabraham]

@cartermp nope, just saw that 15.8 full is out. And with the best will in the world, I just don't have the time to spin up a VM and test out 15.8 from clean - sorry. Either I stick with 15.7 and 15.8 preview (which I've stopped using for now, as 15.7 with the non-nightlies works) or upgrade to 15.8.

[cartermp]

I haven't observed this issue with the 15.8 release. @isaacabraham have you? If not, we can just close this out, since the fix is now in the generally available release.

[isaacabraham]

I don't believe I've had any crashes since upgrading and I'm on the nightly release as well.

[cartermp]

Okay, closing this out. Sounds like the changes we had in 15.8 were the fix here.