-
Notifications
You must be signed in to change notification settings - Fork 842
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SqliteBusy Error appears once in a while #4471
Comments
Do we have multiple threads accessing SQLite at the same time? |
Yes, I do think so. There is also this function inside the module
|
Then that's the issue. SQLite doesn't support multithreaded access to the database. |
@mihaimaruseac why do you think it does not? See e.g. https://www.sqlite.org/threadsafe.html |
I guess that's a new development in slqite. In the past it used to not be possible to properly multithread access the database. |
Another important question to ask: Do we have multiple processes accessing SQLite at the same time? While multi-threaded access isn't a problem for sqlite as linked above, only 1 process can write to a DB at any given time. From https://www.sqlite.org/faq.html#q5:
|
CC @snoyberg via #4247 (comment) |
Right, I get all that. Adding a busy timeout value is intended to give another process a chance (2 seconds) to finish up whatever transaction it's in the middle of. If that's insufficient, it means that Pantry transactions are lasting longer than I would have thought in some cases. |
@snoyberg I don't think there is a guarantee of 2 seconds. From this source:
Reference: http://sqlite.1065341.n5.nabble.com/busy-timeout-not-working-always-tp83035p83037.html |
Also, I have this sample code through which I'm able to reproduce the above SqliteBusy Error (note that you may have to run multiple times to reproduce this): Click to expand the code!
You would have to tweak the code for some path change, but you get the idea. Note that the above code certainly doesn't take 2 seconds to complete - it seems it is upto sqlite's discretion when to return |
I just reproduced this running on Windows 10 while building Stack itself from Git using the command:
|
What's the current status here? I think I remember some additional commits landing to improve this situation, and I personally haven't seen this bug on my machine or CI in a while. Anyone have more recent data? |
@snoyberg This is from yesterday's azure log: https://dev.azure.com/commercialhaskell/stack/_build/results?buildId=944 |
Thanks Sibi, looking at this now. |
This makes two basic changes to the code: * Avoids a connection pool, which forces the SQLite connection to be closed at the end of each call to withStorage. This seems unfortunate; we would like to be able to maintain connections. However, SQLite seems to view such connections as holding a write lock even when not in the middle of a database transaction. * Uses an explicit file lock on the two longest-running write operations: migration and Hackage population. Ideally the locking mechanism in SQLite would be sufficient for this, but (1) Hackage population can take a long time, and (2) evidence has shown that it doesn't behave as we expect it to. This may be insufficient for preventing the bug described in #4471, in which case we may need to apply withWriteLock to more calls. However, I'd like to start off small to avoid the additional overhead.
This makes two basic changes to the code: * Avoids a connection pool, which forces the SQLite connection to be closed at the end of each call to withStorage. This seems unfortunate; we would like to be able to maintain connections. However, SQLite seems to view such connections as holding a write lock even when not in the middle of a database transaction. * Uses an explicit file lock on the two longest-running write operations: migration and Hackage population. Ideally the locking mechanism in SQLite would be sufficient for this, but (1) Hackage population can take a long time, and (2) evidence has shown that it doesn't behave as we expect it to. This may be insufficient for preventing the bug described in #4471, in which case we may need to apply withWriteLock to more calls. However, I'd like to start off small to avoid the additional overhead.
This makes two basic changes to the code: * Avoids a connection pool, which forces the SQLite connection to be closed at the end of each call to withStorage. This seems unfortunate; we would like to be able to maintain connections. However, SQLite seems to view such connections as holding a write lock even when not in the middle of a database transaction. * Uses an explicit file lock on the two longest-running write operations: migration and Hackage population. Ideally the locking mechanism in SQLite would be sufficient for this, but (1) Hackage population can take a long time, and (2) evidence has shown that it doesn't behave as we expect it to. This may be insufficient for preventing the bug described in #4471, in which case we may need to apply withWriteLock to more calls. However, I'd like to start off small to avoid the additional overhead.
This makes two basic changes to the code: * Avoids a connection pool, which forces the SQLite connection to be closed at the end of each call to withStorage. This seems unfortunate; we would like to be able to maintain connections. However, SQLite seems to view such connections as holding a write lock even when not in the middle of a database transaction. * Uses an explicit file lock on the two longest-running write operations: migration and Hackage population. Ideally the locking mechanism in SQLite would be sufficient for this, but (1) Hackage population can take a long time, and (2) evidence has shown that it doesn't behave as we expect it to. This may be insufficient for preventing the bug described in #4471, in which case we may need to apply withWriteLock to more calls. However, I'd like to start off small to avoid the additional overhead.
This makes two basic changes to the code: * Avoids a connection pool, which forces the SQLite connection to be closed at the end of each call to withStorage. This seems unfortunate; we would like to be able to maintain connections. However, SQLite seems to view such connections as holding a write lock even when not in the middle of a database transaction. * Uses an explicit file lock on the two longest-running write operations: migration and Hackage population. Ideally the locking mechanism in SQLite would be sufficient for this, but (1) Hackage population can take a long time, and (2) evidence has shown that it doesn't behave as we expect it to. This may be insufficient for preventing the bug described in #4471, in which case we may need to apply withWriteLock to more calls. However, I'd like to start off small to avoid the additional overhead.
This makes two basic changes to the code: * Avoids a connection pool, which forces the SQLite connection to be closed at the end of each call to withStorage. This seems unfortunate; we would like to be able to maintain connections. However, SQLite seems to view such connections as holding a write lock even when not in the middle of a database transaction. * Uses an explicit file lock on the two longest-running write operations: migration and Hackage population. Ideally the locking mechanism in SQLite would be sufficient for this, but (1) Hackage population can take a long time, and (2) evidence has shown that it doesn't behave as we expect it to. This may be insufficient for preventing the bug described in #4471, in which case we may need to apply withWriteLock to more calls. However, I'd like to start off small to avoid the additional overhead.
This makes two basic changes to the code: * Avoids a connection pool, which forces the SQLite connection to be closed at the end of each call to withStorage. This seems unfortunate; we would like to be able to maintain connections. However, SQLite seems to view such connections as holding a write lock even when not in the middle of a database transaction. * Uses an explicit file lock on the two longest-running write operations: migration and Hackage population. Ideally the locking mechanism in SQLite would be sufficient for this, but (1) Hackage population can take a long time, and (2) evidence has shown that it doesn't behave as we expect it to. This may be insufficient for preventing the bug described in #4471, in which case we may need to apply withWriteLock to more calls. However, I'd like to start off small to avoid the additional overhead.
This makes two basic changes to the code: * Avoids a connection pool, which forces the SQLite connection to be closed at the end of each call to withStorage. This seems unfortunate; we would like to be able to maintain connections. However, SQLite seems to view such connections as holding a write lock even when not in the middle of a database transaction. * Uses an explicit file lock on the two longest-running write operations: migration and Hackage population. Ideally the locking mechanism in SQLite would be sufficient for this, but (1) Hackage population can take a long time, and (2) evidence has shown that it doesn't behave as we expect it to. This may be insufficient for preventing the bug described in #4471, in which case we may need to apply withWriteLock to more calls. However, I'd like to start off small to avoid the additional overhead.
This makes two basic changes to the code: * Avoids a connection pool, which forces the SQLite connection to be closed at the end of each call to withStorage. This seems unfortunate; we would like to be able to maintain connections. However, SQLite seems to view such connections as holding a write lock even when not in the middle of a database transaction. * Uses an explicit file lock on the two longest-running write operations: migration and Hackage population. Ideally the locking mechanism in SQLite would be sufficient for this, but (1) Hackage population can take a long time, and (2) evidence has shown that it doesn't behave as we expect it to. This may be insufficient for preventing the bug described in #4471, in which case we may need to apply withWriteLock to more calls. However, I'd like to start off small to avoid the additional overhead.
This makes two basic changes to the code: * Avoids a connection pool, which forces the SQLite connection to be closed at the end of each call to withStorage. This seems unfortunate; we would like to be able to maintain connections. However, SQLite seems to view such connections as holding a write lock even when not in the middle of a database transaction. * Uses an explicit file lock on the two longest-running write operations: migration and Hackage population. Ideally the locking mechanism in SQLite would be sufficient for this, but (1) Hackage population can take a long time, and (2) evidence has shown that it doesn't behave as we expect it to. This may be insufficient for preventing the bug described in #4471, in which case we may need to apply withWriteLock to more calls. However, I'd like to start off small to avoid the additional overhead.
This makes two basic changes to the code: * Avoids a connection pool, which forces the SQLite connection to be closed at the end of each call to withStorage. This seems unfortunate; we would like to be able to maintain connections. However, SQLite seems to view such connections as holding a write lock even when not in the middle of a database transaction. * Uses an explicit file lock on the two longest-running write operations: migration and Hackage population. Ideally the locking mechanism in SQLite would be sufficient for this, but (1) Hackage population can take a long time, and (2) evidence has shown that it doesn't behave as we expect it to. This may be insufficient for preventing the bug described in #4471, in which case we may need to apply withWriteLock to more calls. However, I'd like to start off small to avoid the additional overhead.
…lock Take a write lock before using Pantry database #4471
#4688 has landed. The integration tests now pass completely, but that's probably more a result of rewriting the test runner than any changes I made to the SQLite handling. I'm going to leave this issue open for now to make sure we revisit this before a release. If anyone who has been affected by this can try upgrading to latest master and reporting whether or not they see this problem anymore, that would be greatly appreciated. |
Take a lock on the Pantry database for all actions (fixes #4471)
Sqlite Busy error still appears
Related issue: #4247
Steps to reproduce
Unfortunately, there are no steps to reproduce it reliable. The Azure pipeline has been able to replicate it periodically:
I was sparsely able to reproduce it twice by running Stack build simultaneously on multiple Haskell projects (on a Linux machine).
Expected
Not have these errors:
Actual
Hitting up with the
ErrorBusy
error.Stack version
Method of installation
From the source
The text was updated successfully, but these errors were encountered: