Port pismo replay tool improvements

[mhofman]

refs: #6723

Description

This is a port of the replay tool changes only of #6723, adapted for the changes in master since the pismo fork.
The changes to the extract transcript tools needed more independent work and were extracted into a separate PR: #7434.
This does include some changes layered on top of the rebase, to make the replay tool working correctly with the new snapstore abstraction, and support the new bundle handling.

Changes compared to original PR

Diff of the ported commits vs original PR:

--- original.diff	2023-04-17 14:55:26.487024034 +0000
+++ port.diff	2023-04-17 14:54:53.924963016 +0000
@@ -1,123 +1,68 @@
-diff --git a/packages/SwingSet/misc-tools/extract-transcript-from-kerneldb.js b/packages/SwingSet/misc-tools/extract-transcript-from-kerneldb.js
-index 107dcc704..528337fe7 100644
---- a/packages/SwingSet/misc-tools/extract-transcript-from-kerneldb.js
-+++ b/packages/SwingSet/misc-tools/extract-transcript-from-kerneldb.js
-@@ -54,6 +54,9 @@ if (!vatName) {
-       )} ${len.padStart(10)} deliveries`,
-     );
-   }
-+} else if (/(supervisor|lockdown)(B|-b)undle/.test(vatName)) {
-+  const bundle = kvStore.get(vatName.replace('-bundle', 'Bundle'));
-+  fs.writeFileSync(vatName.replace('Bundle', '-bundle'), bundle);
- } else {
-   let vatID = vatName;
-   if (allVatNames.indexOf(vatName) !== -1) {
-diff --git a/packages/SwingSet/misc-tools/extract-transcript-from-slogfile.js b/packages/SwingSet/misc-tools/extract-transcript-from-slogfile.js
-index abdae8ad9..3674b83d6 100644
---- a/packages/SwingSet/misc-tools/extract-transcript-from-slogfile.js
-+++ b/packages/SwingSet/misc-tools/extract-transcript-from-slogfile.js
-@@ -47,7 +47,6 @@ async function run() {
-           vatParameters,
-           vatSourceBundle,
-         };
--        transcriptNum += 1;
-         // first line of transcript is the source bundle
-         fs.writeSync(fd, JSON.stringify(t));
-         fs.writeSync(fd, '\n');
-@@ -72,8 +71,8 @@ async function run() {
-       }
-       case 'deliver-result': {
-         console.log(` -- deliver-result`);
--        const entry = { transcriptNum, d: delivery, syscalls };
-         transcriptNum += 1;
-+        const entry = { transcriptNum, d: delivery, syscalls };
-         fs.writeSync(fd, JSON.stringify(entry));
-         fs.writeSync(fd, '\n');
-         break;
-@@ -82,7 +81,6 @@ async function run() {
-         console.log(' -- heap-snapshot-save');
-         const { type, snapshotID } = e;
-         const t = { transcriptNum, type, vatID, snapshotID };
--        transcriptNum += 1;
-         fs.writeSync(fd, JSON.stringify(t));
-         fs.writeSync(fd, '\n');
-         break;
 diff --git a/packages/SwingSet/misc-tools/replay-transcript.js b/packages/SwingSet/misc-tools/replay-transcript.js
-index f77a0a52a..2893270f8 100644
+index b0ffab84a..2df8c3318 100644
 --- a/packages/SwingSet/misc-tools/replay-transcript.js
 +++ b/packages/SwingSet/misc-tools/replay-transcript.js
 @@ -1,3 +1,5 @@
 +// @ts-check
 +
  /* global WeakRef FinalizationRegistry */
  /* eslint-disable no-constant-condition */
  import fs from 'fs';
-@@ -14,6 +16,8 @@ import { pipeline } from 'stream';
+@@ -13,6 +15,10 @@ import { pipeline } from 'stream';
  import { performance } from 'perf_hooks';
  // eslint-disable-next-line import/no-extraneous-dependencies
  import { file as tmpFile, tmpName } from 'tmp';
 +// eslint-disable-next-line import/no-extraneous-dependencies
++import sqlite3 from 'better-sqlite3';
++// eslint-disable-next-line import/no-extraneous-dependencies
 +import yargsParser from 'yargs-parser';
  import bundleSource from '@endo/bundle-source';
  import { makeMeasureSeconds } from '@agoric/internal';
  import { makeSnapStore } from '@agoric/swing-store';
-@@ -28,28 +32,124 @@ import { makeDummyMeterControl } from '../src/kernel/dummyMeterControl.js';
+@@ -27,22 +33,115 @@ import { makeDummyMeterControl } from '../src/kernel/dummyMeterControl.js';
  import { makeGcAndFinalize } from '../src/lib-nodejs/gc-and-finalize.js';
  import engineGC from '../src/lib-nodejs/engine-gc.js';
  
--// Set the absolute path of the SDK to use for bundling
--// This can help if there are symlinks in the path that should be respected
--// to match the path of the SDK that produced the initial transcript
--// For e.g. set to '/src' if replaying a docker based loadgen transcript
--const ABSOLUTE_SDK_PATH = null;
-+const pipe = promisify(pipeline);
-+
-+// TODO: switch to full yargs for documenting output
-+const argv = yargsParser(process.argv.slice(2), {
-+  string: [
-+    // Set the absolute path of the SDK to use for bundling
-+    // This can help if there are symlinks in the path that should be respected
-+    // to match the path of the SDK that produced the initial transcript
-+    // For e.g. set to '/src' if replaying a docker based loadgen transcript
-+    'absoluteSdkPath',
-+  ],
- 
 -// Rebuild the bundles when starting the replay.
 -// Disable if bundles were previously extracted form a Kernel DB, or
 -// to save a few seconds and rely upon previously built versions instead
 -const REBUILD_BUNDLES = false;
++const pipe = promisify(pipeline);
+ 
+-// Enable to continue if snapshot hash doesn't match transcript
+-const IGNORE_SNAPSHOT_HASH_DIFFERENCES = false;
++// TODO: switch to full yargs for documenting output
++const argv = yargsParser(process.argv.slice(2), {
 +  boolean: [
 +    // Rebuild the bundles when starting the replay.
 +    // Disable if bundles were previously extracted form a Kernel DB, or to
 +    // save a few seconds and rely upon previously built versions instead.
 +    'rebuildBundles',
  
--// Enable to continue if snapshot hash doesn't match transcript
--const IGNORE_SNAPSHOT_HASH_DIFFERENCES = false;
+-// Use a simplified snapstore which derives the snapshot filename from the
+-// transcript and doesn't compress the snapshot
+-const USE_CUSTOM_SNAP_STORE = true;
 +    // Enable to continue if snapshot hash doesn't match hash in transcript's
 +    // 'save', or when the hash of the concurrent workers snapshots don't all
 +    // match each other.
 +    'ignoreSnapshotHashDifference',
  
--// Use a simplified snapstore which derives the snapshot filename from the
--// transcript and doesn't compress the snapshot
--const USE_CUSTOM_SNAP_STORE = true;
+-// Enable to output xsnap debug traces corresponding to the transcript replay
+-const RECORD_XSNAP_TRACE = false;
 +    // Enable to continue if concurrent workers do not produce the exact same
 +    // set of syscalls for a delivery. With special virtual collection syscall
 +    // handling (see below), all workers would normally have to diverge from
 +    // the transcript in the same way for the delivery to be considered valid.
 +    'ignoreConcurrentWorkerDivergences',
  
--// Enable to output xsnap debug traces corresponding to the transcript replay
--const RECORD_XSNAP_TRACE = false;
+-const pipe = promisify(pipeline);
 +    // If a snapshot of a worker is taken, create a new worker from that
 +    // snapshot, even if no explicit snapshot load instruction is found in the
 +    // input transcript.
 +    'forcedReloadFromSnapshot',
- 
--const pipe = promisify(pipeline);
++
 +    // Mark workers loaded from an explicit transcript load instruction as
 +    // being ineligible from being reaped.
 +    'keepWorkerExplicitLoad',
 +
 +    // When `forcedReloadFromSnapshot` is enabled, if the hash of the/ snapshot
@@ -170,11 +115,10 @@
 +      key: 'keepWorkerTransactionNums',
 +      number: true,
 +    },
 +  ],
 +  default: {
-+    absoluteSdkPath: '',
 +    rebuildBundles: false,
 +    ignoreSnapshotHashDifference: true,
 +    ignoreConcurrentWorkerDivergences: true,
 +    forcedSnapshotInitial: 2,
 +    forcedSnapshotInterval: 1000,
@@ -201,20 +145,11 @@
 +  },
 +});
  
  /** @type {(filename: string) => Promise<string>} */
  async function fileHash(filename) {
-@@ -78,7 +178,7 @@ function makeSnapStoreIO() {
- async function makeBundles() {
-   const controllerUrl = new URL(
-     `${
--      ABSOLUTE_SDK_PATH ? `${ABSOLUTE_SDK_PATH}/packages/SwingSet` : '..'
-+      argv.absoluteSdkPath ? `${argv.absoluteSdkPath}/packages/SwingSet` : '..'
-     }/src/controller/initializeSwingset.js`,
-     import.meta.url,
-   );
-@@ -96,58 +196,64 @@ async function makeBundles() {
+@@ -78,58 +177,74 @@ async function makeBundles() {
    console.log(`xs bundles written`);
  }
  
 -function compareSyscalls(vatID, originalSyscall, newSyscall) {
 -  const error = requireIdentical(vatID, originalSyscall, newSyscall);
@@ -233,16 +168,16 @@
 +/** @type {import('../src/types-external.js').ManagerType} */
  const worker = 'xs-worker';
  
  async function replay(transcriptFile) {
    let vatID; // we learn this from the first line of the transcript
-+  /** @type {import('../src/types-external.js').VatManagerFactory} */
++  /** @type {import('../src/types-internal.js').VatManagerFactory} */
    let factory;
  
    let loadSnapshotID = null;
    let saveSnapshotID = null;
-+  let lastTranscriptNum;
++  let lastTranscriptNum = 0;
 +  let startTranscriptNum;
    const snapshotOverrideMap = new Map();
  
 -  const fakeKernelKeeper = {
 -    provideVatKeeper: _vatID => ({
@@ -256,10 +191,12 @@
 -    delivery: () => () => undefined,
 -    syscall: () => () => undefined,
 -  };
 -  const snapStore = USE_CUSTOM_SNAP_STORE
 -    ? {
+-        async save(saveRaw) {
+-          const snapFile = `${saveSnapshotID || 'unknown'}.xss`;
 +  const snapshotActivityFd = fs.openSync('snapshot-activity.jsonl', 'a');
 +
 +  const fakeKernelKeeper =
 +    /** @type {import('../src/types-external.js').KernelKeeper} */ ({
 +      provideVatKeeper: _vatID =>
@@ -282,35 +219,46 @@
 +      })
 +    );
 +
 +  const snapStore = argv.useCustomSnapStore
 +    ? /** @type {SnapStore} */ ({
-         async save(saveRaw) {
-           const snapFile = `${saveSnapshotID || 'unknown'}.xss`;
++        async saveSnapshot(_vatID, endPos, saveRaw) {
++          const snapFile = `${vatID}-${endPos}-${
++            saveSnapshotID || 'unknown'
++          }.xss`;
            await saveRaw(snapFile);
 -          const h = await fileHash(snapFile);
 -          await fs.promises.rename(snapFile, `${h}.xss`);
 -          return h;
 +          const hash = await fileHash(snapFile);
-+          const filePath = `${hash}.xss`;
++          const filePath = `${vatID}-${endPos}-${hash}.xss`;
 +          await fs.promises.rename(snapFile, filePath);
-+          return { hash, filePath };
++          return { hash };
          },
-         async load(hash, loadRaw) {
+-        async load(hash, loadRaw) {
++        async loadSnapshot(hash, loadRaw) {
            const snapFile = `${hash}.xss`;
            return loadRaw(snapFile);
          },
 -      }
--    : makeSnapStore(process.cwd(), makeSnapStoreIO());
+-    : makeSnapStore(process.cwd(), () => {}, makeSnapStoreIO());
 -  const testLog = undefined;
 +      })
-+    : makeSnapStore(process.cwd(), makeSnapStoreIO(), { keepSnapshots: true });
++    : makeSnapStore(
++        sqlite3(':memory:'),
++        () => {},
++        makeSnapStoreIO(),
++        undefined,
++        {
++          keepSnapshots: true,
++        },
++      );
 +  const testLog = () => {};
    const meterControl = makeDummyMeterControl();
    const gcTools = harden({
      WeakRef,
-@@ -156,36 +262,63 @@ async function replay(transcriptFile) {
+@@ -138,33 +253,59 @@ async function replay(transcriptFile) {
      gcAndFinalize: makeGcAndFinalize(engineGC),
      meterControl,
    });
 -  const allVatPowers = { testLog };
 -  let xsnapPID;
@@ -320,11 +268,11 @@
 +        testLog,
 +      })
 +    );
 +  /**
 +   * @typedef {{
-+   *  manager: import('../src/types-external.js').VatManager;
++   *  manager: import('../src/types-internal.js').VatManager;
 +   *  xsnapPID: number | undefined;
 +   *  deliveryTimeTotal: number;
 +   *  deliveryTimeSinceLastSnapshot: number;
 +   *  loadSnapshotID: string | undefined;
 +   *  timeOfLastCommand: number;
@@ -349,19 +297,10 @@
 -      JSON.parse(fs.readFileSync('lockdown-bundle')),
 -      JSON.parse(fs.readFileSync('supervisor-bundle')),
 +      JSON.parse(fs.readFileSync('lockdown-bundle', 'utf-8')),
 +      JSON.parse(fs.readFileSync('supervisor-bundle', 'utf-8')),
      ];
--    const env = {};
--    if (RECORD_XSNAP_TRACE) {
-+    const env = /** @type {Record<string, string>} */ ({});
-+    if (argv.recordXsnapTrace) {
-       env.XSNAP_TEST_RECORD = process.cwd();
-     }
-+    if (argv.useXsnapDebug) {
-+      env.XSNAP_DEBUG = 'true';
-+    }
  
 -    const capturePIDSpawn = (...args) => {
 -      const child = spawn(...args);
 -      xsnapPID = child.pid;
 -      return child;
@@ -372,35 +311,41 @@
 +        const child = spawn(...args);
 +        workers[workers.length - 1].xsnapPID = child.pid;
 +        return child;
 +      }
 +    );
-     const startXSnap = makeStartXSnap(bundles, {
+     const startXSnap = makeStartXSnap({
        snapStore,
-       env,
        spawn: capturePIDSpawn,
+-      workerTraceRootPath: RECORD_XSNAP_TRACE ? process.cwd() : undefined,
++      debug: argv.useXsnapDebug,
++      workerTraceRootPath: argv.recordXsnapTrace ? process.cwd() : undefined,
+       overrideBundles: bundles,
++      bundleHandler: /** @type {*} */ (undefined),
      });
      factory = makeXsSubprocessFactory({
 +      allVatPowers,
        kernelKeeper: fakeKernelKeeper,
        kernelSlog,
        startXSnap,
-@@ -219,134 +352,694 @@ async function replay(transcriptFile) {
+@@ -182,134 +323,708 @@ async function replay(transcriptFile) {
      throw Error(`unhandled worker type ${worker}`);
    }
  
 +  const [
 +    bestRequireIdentical,
 +    extraSyscall,
 +    missingSyscall,
 +    vcSyscallRE,
 +    supportsRelaxedSyscalls,
 +  ] = await (async () => {
++    /** @type {any} */
 +    const transcriptModule = await import(
 +      '../src/kernel/vat-loader/transcript.js'
 +    );
 +
++    /** @type {RegExp} */
 +    const syscallRE =
 +      transcriptModule.vcSyscallRE || /^vc\.\d+\.\|(?:schemata|label)$/;
 +
 +    if (
 +      typeof transcriptModule.requireIdenticalExceptStableVCSyscalls !==
@@ -413,30 +358,33 @@
 +        syscallRE,
 +        false,
 +      ];
 +    }
 +
++    /** @type {{requireIdenticalExceptStableVCSyscalls: import('../src/kernel/vat-loader/transcript.js').CompareSyscalls}} */
 +    const { requireIdenticalExceptStableVCSyscalls } = transcriptModule;
 +
 +    if (
 +      typeof transcriptModule.extraSyscall === 'symbol' &&
 +      typeof transcriptModule.missingSyscall === 'symbol'
 +    ) {
 +      return [
 +        requireIdenticalExceptStableVCSyscalls,
-+        transcriptModule.extraSyscall,
-+        transcriptModule.missingSyscall,
++        /** @type {symbol} */ (transcriptModule.extraSyscall),
++        /** @type {symbol} */ (transcriptModule.missingSyscall),
 +        syscallRE,
 +        true,
 +      ];
 +    }
 +
++    /** @type {unknown} */
 +    const dynamicExtraSyscall = requireIdenticalExceptStableVCSyscalls(
 +      'vat0',
 +      ['vatstoreGet', 'vc.0.|label'],
 +      ['vatstoreGet', 'ignoreExtraSyscall'],
 +    );
++    /** @type {unknown} */
 +    const dynamicMissingSyscall = requireIdenticalExceptStableVCSyscalls(
 +      'vat0',
 +      ['vatstoreGet', 'ignoreMissingSyscall'],
 +      ['vatstoreGet', 'vc.0.|label'],
 +    );
@@ -554,11 +502,11 @@
 +    workerData.timeOfLastCommand = NaN;
 +    workerData.deliveryTimeTotal += deliveryTime;
 +    workerData.deliveryTimeSinceLastSnapshot += deliveryTime;
 +  };
 +
-+  /** @type {Map<string, VatSyscallResult | undefined>} */
++  /** @type {Map<string, import('@agoric/swingset-liveslots').VatSyscallResult | undefined>} */
 +  const knownVCSyscalls = new Map();
 +
 +  /**
 +   * @param {import('../src/types-external.js').VatSyscallObject} vso
 +   */
@@ -598,18 +546,26 @@
 +      if (error) {
 +        console.error(
 +          `during transcript num= ${lastTranscriptNum} for worker PID ${workerData.xsnapPID} (start delivery ${workerData.firstTranscriptNum})`,
 +        );
 +
-+        if (error === extraSyscall && !argv.skipExtraVcSyscalls) {
++        if (
++          // @ts-expect-error may be a symbol in some versions
++          error === extraSyscall &&
++          !argv.skipExtraVcSyscalls
++        ) {
 +          return new Error('Extra syscall disallowed');
 +        }
 +      }
 +
 +      const newSyscallKind = newSyscall[0];
 +
-+      if (error === missingSyscall && !argv.simulateVcSyscalls) {
++      if (
++        // @ts-expect-error may be a symbol in some versions
++        error === missingSyscall &&
++        !argv.simulateVcSyscalls
++      ) {
 +        return new Error('Missing syscall disallowed');
 +      }
 +
 +      if (
 +        argv.simulateVcSyscalls &&
@@ -692,12 +648,12 @@
 -    manager = await factory.createFromBundle(
 +    completeWorkerStep(workerData);
 +    workers.push(workerData);
 +    updateWorkersSynced();
 +    const managerOptions =
-+      /** @type {import('../src/types-external.js').ManagerOptions} */ (
-+        /** @type {Partial<import('../src/types-external.js').ManagerOptions>} */ ({
++      /** @type {import('../src/types-internal.js').ManagerOptions} */ (
++        /** @type {Partial<import('../src/types-internal.js').ManagerOptions>} */ ({
 +          sourcedConsole: console,
 +          vatParameters,
 +          compareSyscalls: makeCompareSyscalls(workerData),
 +          useTranscript: true,
 +        })
@@ -936,10 +892,11 @@
 +        /** @param {WorkerData} workerData */
 +        const doWorkerSnapshot = async workerData => {
 +          const { manager, xsnapPID, firstTranscriptNum } = workerData;
 +          if (!manager.makeSnapshot) return null;
 +          const { hash, rawSaveSeconds } = await manager.makeSnapshot(
++            lastTranscriptNum,
 +            snapStore,
 +          );
 +          fs.writeSync(
 +            snapshotActivityFd,
 +            `${JSON.stringify({
@@ -977,11 +934,11 @@
 +          ? workers.reduce(
 +              async (hashes, workerData) => [
 +                ...(await hashes),
 +                await doWorkerSnapshot(workerData),
 +              ],
-+              Promise.resolve(/** @type {string[]} */ ([])),
++              Promise.resolve(/** @type {(string| null)[]} */ ([])),
 +            )
 +          : Promise.all(workers.map(doWorkerSnapshot)));
 +        saveSnapshotID = null;
  
 -  lines.close();
@@ -1047,11 +1004,11 @@
 +            // console.log(`dr`, dr);
 +
 +            // enable this to write periodic snapshots, for #5975 leak
 +            if (makeSnapshot && manager.makeSnapshot) {
 +              const { hash: snapshotID, rawSaveSeconds } =
-+                await manager.makeSnapshot(snapStore);
++                await manager.makeSnapshot(transcriptNum, snapStore);
 +              fs.writeSync(
 +                snapshotActivityFd,
 +                `${JSON.stringify({
 +                  transcriptFile,
 +                  type: 'save',
@@ -1175,45 +1132,45 @@
 +run().catch(err => {
 +  console.log('RUN ERR', err);
 +  process.exit(process.exitCode || 1);
 +});
 diff --git a/packages/SwingSet/package.json b/packages/SwingSet/package.json
-index cba2e5cf3..8f71b62e1 100644
+index f0419d349..16104e844 100644
 --- a/packages/SwingSet/package.json
 +++ b/packages/SwingSet/package.json
-@@ -22,7 +22,8 @@
-   },
-   "devDependencies": {
+@@ -25,7 +25,8 @@
+     "@types/microtime": "^2.1.0",
      "@types/tmp": "^0.2.0",
+     "better-sqlite3": "^8.2.0",
 -    "tmp": "^0.2.1"
 +    "tmp": "^0.2.1",
-+    "yargs-parser": "^21.0.0"
++    "yargs-parser": "^21.1.1"
    },
    "dependencies": {
      "@agoric/assert": "^0.5.1",
 diff --git a/packages/SwingSet/src/kernel/vat-loader/manager-helper.js b/packages/SwingSet/src/kernel/vat-loader/manager-helper.js
-index e12bbee44..dcbbaa076 100644
+index e343bea46..fbe0d28b2 100644
 --- a/packages/SwingSet/src/kernel/vat-loader/manager-helper.js
 +++ b/packages/SwingSet/src/kernel/vat-loader/manager-helper.js
-@@ -103,7 +103,7 @@ import { makeTranscriptManager } from './transcript.js';
+@@ -109,7 +109,7 @@ import { makeTranscriptManager } from './transcript.js';
   * @param {KernelSlog} kernelSlog
   * @param {(vso: VatSyscallObject) => VatSyscallResult} vatSyscallHandler
   * @param {boolean} workerCanBlock
-- * @param {(vatID: any, originalSyscall: any, newSyscall: any) => import('./transcript.js').CompareSyscallsResult} [compareSyscalls]
+- * @param {(vatID: any, originalSyscall: any, newSyscall: any) => Error | undefined} [compareSyscalls]
 + * @param {import('./transcript.js').CompareSyscalls} [compareSyscalls]
   * @param {boolean} [useTranscript]
   * @returns {ManagerKit}
   */
-@@ -119,6 +119,7 @@ function makeManagerKit(
+@@ -125,6 +125,7 @@ function makeManagerKit(
  ) {
    assert(kernelSlog);
    const vatKeeper = kernelKeeper.provideVatKeeper(vatID);
 +  /** @type {ReturnType<typeof makeTranscriptManager> | undefined} */
    let transcriptManager;
    if (useTranscript) {
      transcriptManager = makeTranscriptManager(
-@@ -236,7 +237,6 @@ function makeManagerKit(
+@@ -238,7 +239,6 @@ function makeManagerKit(
     * just direct).
     *
     * @param {VatSyscallObject} vso
 -   * @returns {VatSyscallResult}
     */
@@ -1221,129 +1178,94 @@
      if (transcriptManager && transcriptManager.inReplay()) {
 @@ -267,7 +267,7 @@ function makeManagerKit(
    /**
     *
     * @param { () => Promise<void>} shutdown
--   * @param {(ss: SnapStore) => Promise<SnapshotInfo>} makeSnapshot
-+   * @param {(ss: SnapStore) => Promise<SnapshotInfo>} [makeSnapshot]
+-   * @param {(endPos: number, ss: SnapStore) => Promise<SnapshotResult>} makeSnapshot
++   * @param {(endPos: number, ss: SnapStore) => Promise<SnapshotResult>} [makeSnapshot]
     * @returns {VatManager}
     */
    function getManager(shutdown, makeSnapshot) {
 diff --git a/packages/SwingSet/src/kernel/vat-loader/transcript.js b/packages/SwingSet/src/kernel/vat-loader/transcript.js
-index de5f0c892..1acb0e614 100644
+index e90100fea..45346671f 100644
 --- a/packages/SwingSet/src/kernel/vat-loader/transcript.js
 +++ b/packages/SwingSet/src/kernel/vat-loader/transcript.js
-@@ -1,20 +1,30 @@
+@@ -1,5 +1,25 @@
 +// @ts-check
 +
  import djson from '../../lib/djson.js';
  
- // Indicate that a syscall is missing from the transcript but is safe to
- // perform during replay
--const missingSyscall = Symbol('missing transcript syscall');
-+export const missingSyscall = Symbol('missing transcript syscall');
- 
- // Indicate that a syscall is recorded in the transcript but can be safely
- // ignored / skipped during replay.
--const extraSyscall = Symbol('extra transcript syscall');
-+export const extraSyscall = Symbol('extra transcript syscall');
- 
- /** @typedef {typeof missingSyscall | typeof extraSyscall | Error | undefined} CompareSyscallsResult */
++/** @typedef {import('@agoric/swingset-liveslots').VatSyscallObject} VatSyscallObject */
++/** @typedef {import('@agoric/swingset-liveslots').VatSyscallResult} VatSyscallResult */
++
++/** @typedef {Error | undefined} CompareSyscallsResult */
 +/**
 + * @typedef {(
 + *     vatId: any,
 + *     originalSyscall: VatSyscallObject,
 + *     newSyscall: VatSyscallObject,
-+ *     originalResponse?: VatSyscallResult,
 + *   ) => CompareSyscallsResult
 + * } CompareSyscalls
 + */
- 
- /**
-  * @param {any} vatID
-- * @param {object} originalSyscall
-- * @param {object} newSyscall
-- * @returns {CompareSyscallsResult}
++
++/**
++ * @param {any} vatID
 + * @param {VatSyscallObject} originalSyscall
 + * @param {VatSyscallObject} newSyscall
-  */
++ */
  export function requireIdentical(vatID, originalSyscall, newSyscall) {
    if (djson.stringify(originalSyscall) !== djson.stringify(newSyscall)) {
-@@ -26,7 +36,7 @@ export function requireIdentical(vatID, originalSyscall, newSyscall) {
+     console.error(`anachrophobia strikes vat ${vatID}`);
+@@ -10,6 +30,11 @@ export function requireIdentical(vatID, originalSyscall, newSyscall) {
    return undefined;
  }
  
--const vcSyscallRE = /^vc\.\d+\.\|(?:schemata|label)$/;
-+export const vcSyscallRE = /^vc\.\d+\.\|(?:schemata|label)$/;
- 
- /**
-  * Liveslots currently has a deficiency which results in [virtual collections
-@@ -54,8 +64,8 @@ const vcSyscallRE = /^vc\.\d+\.\|(?:schemata|label)$/;
-  * `simulateSyscall` which then performs the appropriate action.
-  *
-  * @param {any} vatID
-- * @param {object} originalSyscall
-- * @param {object} newSyscall
-+ * @param {VatSyscallObject} originalSyscall
-+ * @param {VatSyscallObject} newSyscall
-  * @returns {CompareSyscallsResult}
-  */
- export function requireIdenticalExceptStableVCSyscalls(
-@@ -87,6 +97,11 @@ export function requireIdenticalExceptStableVCSyscalls(
-   return error;
- }
- 
 +/**
 + * @param {*} vatKeeper
 + * @param {*} vatID
 + * @param {CompareSyscalls} compareSyscalls
 + */
  export function makeTranscriptManager(
    vatKeeper,
    vatID,
-@@ -135,12 +150,14 @@ export function makeTranscriptManager(
+@@ -58,7 +83,9 @@ export function makeTranscriptManager(
  
    let replayError;
  
 +  /** @param {VatSyscallObject} newSyscall */
    function simulateSyscall(newSyscall) {
-     while (playbackSyscalls.length) {
-       const compareError = compareSyscalls(
-         vatID,
-         playbackSyscalls[0].d,
-         newSyscall,
-+        playbackSyscalls[0].response,
-       );
- 
-       if (compareError === missingSyscall) {
-@@ -149,6 +166,7 @@ export function makeTranscriptManager(
-         return undefined;
-       }
- 
-+      /** @type {{d: VatSyscallObject; response: VatSyscallResult}} */
-       const s = playbackSyscalls.shift();
- 
-       if (!compareError) {
-diff --git a/packages/SwingSet/src/types-external.js b/packages/SwingSet/src/types-external.js
-index 629fa9164..18b177471 100644
---- a/packages/SwingSet/src/types-external.js
-+++ b/packages/SwingSet/src/types-external.js
-@@ -40,7 +40,7 @@ export {};
-  *   vatParameters: Record<string, unknown>,
-  *   virtualObjectCacheSize: number,
++    /** @type {{d: VatSyscallObject; response: VatSyscallResult}} */
+     const s = playbackSyscalls.shift();
+     const newReplayError = compareSyscalls(vatID, s.d, newSyscall);
+     if (newReplayError) {
+diff --git a/packages/SwingSet/src/types-internal.js b/packages/SwingSet/src/types-internal.js
+index dc6b7d716..07a10204e 100644
+--- a/packages/SwingSet/src/types-internal.js
++++ b/packages/SwingSet/src/types-internal.js
+@@ -20,6 +20,7 @@ export {};
+  * @typedef { import('./types-external.js').OptEnableDisavow } OptEnableDisavow
+  * @typedef { import('@agoric/swingset-liveslots').VatDeliveryObject } VatDeliveryObject
+  * @typedef { import('@agoric/swingset-liveslots').VatDeliveryResult } VatDeliveryResult
++ * @typedef { import('@agoric/swingset-liveslots').VatSyscallObject } VatSyscallObject
+  *
+  * // used by vatKeeper.setSourceAndOptions(source, RecordedVatOptions)
+  *
+@@ -34,7 +35,7 @@ export {};
+  *   enableDisavow: boolean,
+  *   useTranscript: boolean,
   *   name: string,
 - *   compareSyscalls?: (originalSyscall: {}, newSyscall: {}) => Error | undefined,
 + *   compareSyscalls?: import('./kernel/vat-loader/transcript.js').CompareSyscalls,
   *   sourcedConsole: Pick<Console, 'debug' | 'log' | 'info' | 'warn' | 'error'>,
-  *   meterID?: string,
-  * } & (HasBundle | HasSetup)} ManagerOptions
-@@ -222,8 +222,9 @@ export {};
-  *                                 vatSyscallHandler: unknown) => Promise<VatManager>,
-  *            } } VatManagerFactory
+  *   enableSetup: boolean,
+  *   setup?: unknown,
+@@ -42,8 +43,9 @@ export {};
+  * }} ManagerOptions
+  *
   * @typedef { { deliver: (delivery: VatDeliveryObject) => Promise<VatDeliveryResult>,
 + *              replayOneDelivery: (delivery: VatDeliveryObject, expectedSyscalls: VatSyscallObject[], deliveryNum: number) => Promise<VatDeliveryResult>,
-  *              replayTranscript: (startPos: StreamPosition | undefined) => Promise<number?>,
-- *              makeSnapshot?: (ss: SnapStore) => Promise<SnapshotInfo>,
-+ *              makeSnapshot?: undefined | ((ss: SnapStore) => Promise<SnapshotInfo>),
+  *              replayTranscript: (startPos: number | undefined) => Promise<number?>,
+- *              makeSnapshot?: (endPos: number, ss: SnapStore) => Promise<SnapshotResult>,
++ *              makeSnapshot?: undefined | ((endPos: number, ss: SnapStore) => Promise<SnapshotResult>),
   *              shutdown: () => Promise<void>,
   *            } } VatManager
-  *
+  * @typedef { { createFromBundle: (vatID: string,

Security Considerations

None, this only changes internal tools.

Scaling Considerations

None

Documentation Considerations

Updates and fixes a few types

Testing Considerations

Manually verified that the transcripts generated by #7434 when running a modified test-vaults-integration.js could be replayed successfully.

[warner]

LGTM

Anything that lets this work is fine by me, so I didn't look too carefully at replay-transcript.js itself, just the code in src/ to make sure production code isn't impacted.

The one guideline I'd suggest is to think of ways that replay-transcript.js can export a function or two to let us run a unit test on it, just a basic smoketest that makes sure a simple transcript can be replayed without crashing. Not necessary to do now, but something to aim for in the near future.

[warner]

What does it mean to have both the ? in makeSnapsot? and the undefined | ?

The original intention was that manager.makeSnapshot may or may not be present, but if it is, then it must be a function with the given signature.

[mhofman]

I don't remember why I had to do this, but I think tsc was complaining without it back in the original pismo work

I do remember something about makeSnapshot being provided as argument and unconditionally installed in the returned object even if undefined, so this is technically more accurate.