Developer tools: data script replayed on restart problem

[ggsurrel]

I have a spring boot application using liquibase and referencing a startup script using spring.datasource.data property. After migrating to spring boot 1.3.0 and testing developer tools automatic restart, I have a problem as my startup script is replayed at startup but previously loaded data is still there. Is there a way to avoid replaying startup script or to flush data on dev tools restart ?

[snicoll]

How is your application supposed to run without devtools then? If it is being initialized on startup and it breaks that way, it should break if you stop the app and start it again (without Devtools). Can you give us a bit more details?

[ggsurrel]

I have made a minimal project to reproduce the problem here: https://github.com/gsurrel-orange/boot-liquibase-dev-tools. On the first launch everything works fine but when the app is restarted by the dev tool module, there is a problem re-running the import script.

[philwebb]

@gsurrel-orange Are you using an in-memory database? Is the root cause of this that a devtools restart doesn't clear the database where as a full restart does?

[ggsurrel]

@philwebb Exactly. This is why I can imagine 2 solutions: either clear the database like it's done on a full restart or avoid replaying the script.

[philwebb]

I think clearing the database would be the better option, I'm actually quite surprised that doesn't happen by default. Thanks for reporting, will take a look.

[wilkinsona]

tomcat-jdbc is on the classpath so while an embedded database is being used, Boot's running in non-embedded database mode. That means that the embedded database isn't closed when the application context closes.

[wilkinsona]

@gsurrel-orange A workaround until we fix this properly is to exclude tomcat-jdbc:

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-jpa</artifactId>
            <exclusions>
                <exclusion>
                    <groupId>org.apache.tomcat</groupId>
                    <artifactId>tomcat-jdbc</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

[wilkinsona]

This isn't as simple as I'd hoped. Closing the connection pool DataSource (Hikari, Tomcat JDBC, Commons DBCP, or Commons DBCP2) isn't sufficient as that doesn't close/clean up the underlying embedded database. In fact, Hikari and Commons DBCP 2 are already closed automatically as they implement AutoCloseable.

[wilkinsona]

@gsurrel-orange Another workaround, that allows you to continue to use a connection pool, is to add the following bean to your application:

    @Bean
    @DependsOn("dataSource")
    public DisposableBean embeddedDatabaseShutdownExecutor(DataSource dataSource) {
        return new DisposableBean() {

            @Override
            public void destroy() throws Exception {
                dataSource.getConnection().createStatement().execute("SHUTDOWN");
            }

        };
    }

[ggsurrel]

@wilkinsona Ok, thanks for pointing the workarounds.

[wilkinsona]

There are probably two different goals that we want to support here. Some users will want their in-memory database to be fresh after each restart and others will want its state to be preserved across a restart.

Providing a fresh database is probably the easier of the two, conceptually at least, as all that is necessary is for the database to be shut down when the context closes during the restart. When the new context is refreshed a new database will be created.

Preserving the state of the database is a bit more complicated. How complicated it is depends on what technology, if any, is being used to set up the database during context refresh:

1. data.sql or schema.sql scripts may fail if they haven't been written to cope with existing tables and data
2. Flyway and Liquibase should be fine. They should no-op as all of the migrations/change sets will have already been applied.
3. Hibernate may cause problems if create or create-drop is being used as both will drop the database's tables when the session factory is opened (the latter will also drop them when it's closed)

If we want to support all of these options (I'm not yet sure that we do) it sounds like DevTools requires a switch that allows a user to say whether or not they want the database's state to be preserved.

If they don't want the state to be preserved we need to figure out how to shut down the in-memory database as part of the context being closed.

If they want the state to be preserved then the ideal is for them to use Flyway or Liquibase. In that case we don't have to do anything. To support data.sql or scheme.sql we could set spring.datasource.initialize=false for restarts. To support Hibernate we could set spring.jpa.hibernate.ddl-auto=update for restarts.

Perhaps Boot shouldn't try to be smart about this and, instead, DevTools should allow users to set properties that only apply to restarts? It requires a bit of manual effort, but that may be better than us guessing the user's intentions and getting it wrong.

[wilkinsona]

We decided that, as a first step at least, the best approach here is to align the behaviour of a pooled in-memory DataSource will a plain in-memory DataSource. To that end, DevTools will connect to a pooled in-memory DataSource and execute SHUTDOWN when the context is closed as part of a restart.

The more complex changes discussed above may be implemented in the future, but only if we see a clear need.

[jhyot]

I am interested in setting properties that apply only on restarts. Is there a custom way to achieve this today?
Or in other words, is there a way for a Bean to detect that a restart is under way. Then I think at least that I could inject specific settings into the environment and so override some properties.

I want to achieve exactly the use case of not running the data sql script on restart, but I agree that it probably doesn't make sense for Spring Boot to be smart about it. So I'd be perfectly happy with any kind of manual way of doing it.

[snicoll]

@jhyot this issue is closed and we prefer to use StackOverflow for questions. Thanks!