Skip to content

Commit

Permalink
fix: export state-sync snapshot without a DB write-lock
Browse files Browse the repository at this point in the history
Exporting a state-sync snapshot is a read-only operation, and is
designed to run "in the background", i.e. in parallel with normal
mutating operations. It accomplishes this by opening a read-only
transaction right away, effectively capturing a snapshot of the SQLite
database state, to insulate the export process from ongoing writes by
the execution host.

The cosmic-swingset exporter starts with a query of `host.height`, to
confirm that the database has not already advanced to a new block
before this snapshot/read-transaction can be taken.

Previously, this query worked by using `openSwingStore`, and then
calling `hostStorage.hostKVStore.get('host.height')`. This had two
problems:

* TOCTTOU: the `hostKVStore.get` used a different DB connection (and
  different txn) than the exporter, so it might return a different
  height, negating the accuracy of the consistency check

* read-write txn: `openSwingStore` creates a read-*write* txn, even
  when merely opening the DB (because it might need to create the
  initial tables). This txn is closed right away, before
  `openSwingStore()` returns, so it did not present a threat to
  ongoing operations. But if the exporter was created while the
  ongoing execution side already had its own read-write txn
  open (e.g. while `controller.run()` was running), then it would
  fail, and `makeSwingStoreExporter` would fail with `SQLITE_BUSY`

Instead, we take advantage of the new `swingStoreExporter.getHostKV()`
API, and use *it* to fetch `host.height`. Unlike the normal
swingstore, the swingstore-exporter refrains from creating read-write
transactions entirely. So the cosmic-swingset export code can safely
query the height without fear of getting the wrong value or failing
because of an ongoing write transaction.

We think this should fix the SQLITE_BUSY errors.

refs #8523
  • Loading branch information
warner authored and mhofman committed Dec 6, 2023
1 parent 43f4e8e commit 06b8c86
Showing 1 changed file with 2 additions and 7 deletions.
9 changes: 2 additions & 7 deletions packages/cosmic-swingset/src/export-kernel-db.js
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ import { makePromiseKit } from '@endo/promise-kit';
import { Fail, q } from '@agoric/assert';
import { makeAggregateError } from '@agoric/internal';
import { makeShutdown } from '@agoric/internal/src/node/shutdown.js';
import { openSwingStore, makeSwingStoreExporter } from '@agoric/swing-store';
import { makeSwingStoreExporter } from '@agoric/swing-store';

import { isEntrypoint } from './helpers/is-entrypoint.js';
import { makeProcessValue } from './helpers/process-value.js';
Expand Down Expand Up @@ -144,7 +144,6 @@ export const validateExporterOptions = options => {
* @param {Pick<import('fs/promises'), 'open' | 'writeFile'>} powers.fs
* @param {import('path')['resolve']} powers.pathResolve
* @param {typeof import('@agoric/swing-store')['makeSwingStoreExporter']} [powers.makeSwingStoreExporter]
* @param {typeof import('@agoric/swing-store')['openSwingStore']} [powers.openSwingStore]
* @param {null | ((...args: any[]) => void)} [powers.log]
* @returns {StateSyncExporter}
*/
Expand All @@ -154,7 +153,6 @@ export const initiateSwingStoreExport = (
fs: { open, writeFile },
pathResolve,
makeSwingStoreExporter: makeExporter = makeSwingStoreExporter,
openSwingStore: openDB = openSwingStore,
log = console.log,
},
) => {
Expand Down Expand Up @@ -183,10 +181,7 @@ export const initiateSwingStoreExport = (
});
cleanup.push(async () => swingStoreExporter.close());

const { hostStorage } = openDB(stateDir);

savedBlockHeight = Number(hostStorage.kvStore.get('host.height')) || 0;
await hostStorage.close();
savedBlockHeight = Number(swingStoreExporter.getHostKV('host.height')) || 0;

if (blockHeight) {
blockHeight === savedBlockHeight ||
Expand Down

0 comments on commit 06b8c86

Please sign in to comment.