Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add synchronization when locking database connection #1200

Merged
merged 2 commits into from
Dec 12, 2019

Conversation

dpad85
Copy link
Member

@dpad85 dpad85 commented Nov 4, 2019

This pull request fixes an issue where creating concurrent database connection could deadlock.

Context

An exclusive lock was added to eclair.sqlite to prevent running several instances of eclair using the same sqlite database. The code obtaining the exclusive lock is a succession of three SQL queries, and is not thread-safe. If two threads try to obtain the lock at the same time, none of these locks will succeed, and subsequent locks will fail because the database is deadlocked.

This can happen on the eclair mobile application where several background jobs (that need exclusive lock on the eclair.sqlite db) may be launched at the exact same time by the OS due to battery optimization.

See:

Proposed change

This PR changes the SqliteUtils.obtainExclusiveLock method into a synchronized statement. Also, if the SQL connections creation fails, db connections will be closed if needed.

@codecov-io
Copy link

Codecov Report

Merging #1200 into master will increase coverage by 0.17%.
The diff coverage is 68.75%.

@@            Coverage Diff             @@
##           master    #1200      +/-   ##
==========================================
+ Coverage   75.26%   75.44%   +0.17%     
==========================================
  Files         135      135              
  Lines        9082     9093      +11     
  Branches      358      363       +5     
==========================================
+ Hits         6836     6860      +24     
+ Misses       2246     2233      -13
Impacted Files Coverage Δ
.../scala/fr/acinq/eclair/db/sqlite/SqliteUtils.scala 94.28% <100%> (+0.16%) ⬆️
.../src/main/scala/fr/acinq/eclair/db/Databases.scala 80% <66.66%> (-20%) ⬇️
...src/main/scala/fr/acinq/eclair/router/Router.scala 91.86% <0%> (+0.72%) ⬆️
...q/eclair/blockchain/electrum/ElectrumWatcher.scala 51.72% <0%> (+0.86%) ⬆️
...nq/eclair/blockchain/electrum/ElectrumClient.scala 73.72% <0%> (+1.45%) ⬆️
...cinq/eclair/blockchain/bitcoind/zmq/ZMQActor.scala 94.87% <0%> (+5.12%) ⬆️
...clair/blockchain/electrum/ElectrumClientPool.scala 78.49% <0%> (+7.52%) ⬆️

@@ -99,7 +99,7 @@ object SqliteUtils {
*
* The lock will be kept until the database is closed, or if the locking mode is explicitly reset.
*/
def obtainExclusiveLock(sqlite: Connection) {
def obtainExclusiveLock(sqlite: Connection) = synchronized {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This only fixes the issue if two calls to obtainExclusiveLock happen in the same process, but in the case of a background job running concurrently with the wallet app, this won't fix anything, right?
It's very surprising that sqlite doesn't have a way to atomically get a real exclusive lock or fail...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This scenario is taken care of by the front application and should not happen. There's a logic that prevents background jobs to run concurrently with the front.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok in that case the synchronized statement should work (and it doesn't hurt to have it anyway).
I still think we can probably do better on the sqlite side with a better understanding of their locking mechanisms, but it can be done later.

}

object Databases {
object Databases extends Logging {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like we should try to avoid extending this Logging trait and instead implicitly pass a logger to the functions that need one. Can you try doing that?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using grizzled.slf4j.Logging is indeed discouraged, but for code completely unrelated to actors I think it makes sense to keep using it. What we don't want is lose the context when we call code in static methods from within an actor.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants