RuntimeError: unreachable when using loadDataDir

[joehan]

Hi, I'm a maintainer of firebase-tools, which has a dependency on pglite. We recently added a import/export feature, where we export Postgres data using dumpDataDir() and then import it on future runs using loadDataDir when we next start up the PGlite database. Some users of this feature have reported the following error when importing data (firebase/firebase-tools#8044):

    at wasm://wasm/0222924a:wasm-function[5668]:0x42d017
    at wasm://wasm/0222924a:wasm-function[4558]:0x3a93a4
    at wasm://wasm/0222924a:wasm-function[2877]:0x2505b8
    at Module._pg_initdb (/Users/joehanley/Workspace/firebase-tools/node_modules/@electric-sql/pglite/dist/index.cjs:12:118235)
    at eE.ua (/Users/joehanley/Workspace/firebase-tools/node_modules/@electric-sql/pglite/dist/index.cjs:12:2428733)
    at async eE.create (/Users/joehanley/Workspace/firebase-tools/node_modules/@electric-sql/pglite/dist/index.cjs:12:2423415)

This only occurs sometimes, and I'm having trouble debugging due to the opaque WASM error.

Reproduction instructions:

Using the following example postgres dump:
postgres.tar.gz

const rf = fs.readFileSync("postgres.tar.gz");
const file = new File([rf], "postgres.tar.gz");
const pgliteArgs: PGliteOptions = {
  username: "blah"
  database: "postgres",
  debug: 0,
  extensions: {
    vector,
    uuidOssp,
  },
  loadDataDir: "postgres.tar.gz",
};

const db = await PGlite.create(pgliteArgs);

In case its relevant, postgres.tar.gz was originally generated via the following code:

    const dump = await db.dumpDataDir();
    const arrayBuff = await dump.arrayBuffer();
    fs.writeFileSync(exportPath, new Uint8Array(arrayBuff));

Any pointers on how to understand this error or ideas on what is wrong with the dumped TAR?

[samwillis]

Hey @joehan, thanks for the report. We'll take a look at it and get back to you asap.

[yacineb]

@joehan can you please tell us which pglite version are you using ? thks

[copiltembel]

@joehan This seems related to loading a dump with a username and database specified. We're looking into it.

[joehan]

@joehan can you please tell us which pglite version are you using ? thks

Sorry, meant to include this in the orignal message. We are currently using 0.2.0 - however, I also tried updating to 0.2.15 and the issue was still occurring.

[joehan]

@joehan This seems related to loading a dump with a username and database specified. We're looking into it.

Do you know specifically what triggers it? If so, we might be able to avoid that situation in the firebase-tools codebase as a immediate workaround.

[copiltembel]

@joehan Could you please try by not specifying the username and database?

[copiltembel]

@joehan Did the above workaround solve it for you?

[joehan]

Sorry for the radio silence - yes, that workaround was successful! Thank you for the help 🙏

[jamesarosen]

It would be lovely to have a more descriptive error message. Even better, PGliteOptions could be something like

type PGliteOptions =
  | { loadDataDir: string; username: never; database: never; }
  | { loadDataDir: never; username?: string; database?: string; }
  ...