Host Recycles causing AccessViolations

[mathewc]

In certain situations the runtime initiates a host shutdown via HostingEnvironment.InitiateShutdown, for example when an unhandled global exception occurs, when a function TimeoutException is thrown, or when performance counter thresholds are exceeded (HostHealthMonitor). This results in a new AppDomain being created in the same process.

When the FunctionApp includes Node.js functions, this AppDomain recycling can trigger the existing known issue reported here, resulting in AccessViolations like the following when a new host is initialized in the new AppDomain:

Description: The process was terminated due to an unhandled exception.
Exception Info: System.AccessViolationException
   at EdgeJs.Edge.NodeStartx86(Int32, System.String[])
   at EdgeJs.Edge+<>c__DisplayClass2.<Func>b__1()
   at System.Threading.ThreadHelper.ThreadStart_Context(System.Object)
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
   at System.Threading.ThreadHelper.ThreadStart()

The issue is that within the process we've already initialized Edge.JS, however when we recycle the AppDomain we try to initialize again, triggering the bug.

Ideally we'd fix the Edge.JS bug - or find a workaround for it. An option until that happens would be for us to change the host recycle logic such that if Node.JS is being used, we do a harder process recycle (e.g. Process.Close()).

Note that these recycles are already happening in situations where an error is forcing the host to restart, so the effect of this bug currently is to just make the host restart process take a bit longer. I.e. an error occurs, we recycle the app domain, when the host spins up it causes an AV which kills the process.

[fabiocav]

Note that this has been around for a while and will ultimately cause the process to crash/restart.

[paulbatum]

Closing this as won't fix since this is a V1 only bug with minimal impact (we can revisit if this behavior ends up causing customer issues in production).

[eHealthManny]

Assuming that @mathewc is correct in his comment on #3074, is there any chance this could be revisited since it can affect end users who bring in the "edge" node module manually?

If not is there a timeline for when V2 will leave preview?

Thanks

[mathewc]

No we won't be revisiting this. In v2 we no longer use EdgeJs as our execution engine for Javascript, so investing time in pushing for an EdgeJs fix for this issue doesn't make sense for us.

No firm date for v2 GA yet - ball park, this fall or end of the year.

[mhoeger]

@eHealthManny - update: V2 is now GA!