Releasing Changelog Locks

When our application comes up, Liquibase will attempt to check for the existence of certain database migrations, and will perform new ones if they have not already been applied. In order to do this, Liquibase must acquire a changelog lock. In essence, all migrations are stored in a specific table in our database; this lock is required to keep multiple Liquibase instances from iterating over the same database.

Unfortunately, if a long-running migration times out, or another issue causes the Java app to crash, Liquibase will not gracefully release the changelog lock. This will result in new application instances spinning up, waiting for 10 minutes to acquire a lock, crashing, and dying. This cycle will continue indefinitely until an engineer manually intervenes.

To verify the presence of a lock, execute the Bastion login procedure. You can find this process in our secure docset.

Once you are logged into the database, run the following query:

select * from public.databasechangeloglock;

You will be met with a response that looks like the following:

id	locked	lockgranted	lockedby
1	TRUE	2021-04-26T22:25:47.304	localhost (127.0.0.1)

To clear the lock, first verify that the application does not hold a legitimate lock. Removing a lock in the middle of an active process may result in data corruption.

Once you have verified that the held lock is orphaned, run the following query to clear it:

update public.databasechangeloglock set locked=false;