Deadlock when receiving first http request while shutting down context. #4130
Comments
magJ
changed the title
|
Thanks very much for the demo application. I've managed to reproduce the problem, both with the versions you were using and with Spring Boot 1.3.0 snapshots on Java 1.8.0 u60. |
|
Here's a thread dump captured using the latest 1.3.0 snapshot: An HTTP request has been received on thread Refresh of the application context has thrown a
|
|
Things are going wrong at the outset due to the fact that we stop the embedded container after we've closed the application context: @Override
protected void doClose() {
super.doClose();
stopAndReleaseEmbeddedServletContainer();
}This means that we can receive a request after the application context has been closed which causes The current start up and shut down ordering is:
I would appear to make more sense for the ordering to be:
|
philwebb
added a commit
that referenced
this issue
Nov 6, 2015
Update EmbeddedWebApplicationContext so that the servlet container is shutdown after the context is closed. Unfortunately shutting the container down before the context has been closed causes exceptions if the `/shutdown` actuator endpoint is used. It can also cause the Tomcat classloader to throw IllegalStateExceptions if resources are accessed during shutdown. As this commit effectively reverts 0069e41 we need to fix the shutdown deadlock issue reported in gh-4130 in a different way. The deadlock can be caused when an incoming HTTP connection occurs whilst the context is closing. The incoming connection triggers the `FrameworkServlet` to call `initWebApplicationContext` which in turn calls `refresh`. The `FrameworkServlet` checks `ApplicationContext.isActive()` before performing an initialization but prior to this commit we would set active to `false` before stopping the servlet container. We now override `onClose` rather than `doClose` in `EmbeddedWebApplicationContext` to ensure that the active flag is only set to `false` once the servlet container has been stopped. See gh-4130 Fixes gh-4396
|
@magJ Our original fix for this caused some other issues so we've needed to change things. I've tested your original deadlock app and it seems fine but if you get a chance could you please try the latest 1.2.8.BUILD-SNAPSHOT version? |
|
Sorry for the late reply, but it seems to work fine with the 1.2.8.BUILD-SNAPSHOT version, thanks. |
|
@magJ Thanks for testing the snapshot. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
If we begin to shutdown the spring context while we receive the first HTTP request, spring-boot can deadlock.
It seems that the first HTTP request kicks off some initialization that grabs a lock on the AnnotationConfigEmbeddedWebApplicationContext.
We saw this in a production application, but I managed to replicate it with some success in a demo application:
https://github.com/magJ/spring-boot-deadlock-issue-4130/
It might take a few attempts, but the above application should deadlock eventually.
Spring boot version: 1.2.4-RELEASE
Java Version: 1.8.0_20
I also have a log sample and thread dump of what we saw in out prod application:
https://gist.github.com/magJ/21b075fe38fdace94be8
The text was updated successfully, but these errors were encountered: