Why does import cause errors in a permissioned upgrade (CoreEval) script? Is there a better way?

[dckc]

These under-documented details of permissioned deployment have come up in work with @Chris-Hibbert on #8018 and with @iomekam on #7954.

cc @turadg @michaelfig

[dckc]

import causes errors in CoreEval scripts because they are evaluated as scripts, not treated as modules. It's very much like this common situation:

• javascript - Node.js: SyntaxError: Cannot use import statement outside a module - Stack Overflow

Detail: compartment.evaluate(code)

The evaluation happens in this code:

agoric-sdk/packages/vats/src/core/chain-behaviors.js

Lines 85 to 89 in ba3b776

	// Evaluate the code in the context of the globals.
	const compartment = new Compartment(globals);
	harden(compartment.globalThis);
	const behavior = compartment.evaluate(code);
	return behavior(powers);

See ses Compartment re .evaluate().

Other ses Compartment limitations: dynamic import

Currently ses has another relevant limitation:

Conservatively reject the source text if it may contain a dynamic
import expression. To reject without parsing, rejectImportExpressions will
also reject some other text as well.
-- rejectImportExpressions

This means the string import( cannot occur anywhere in the script, even in comments.

more on a better way separately...

[dckc]

agoric run supports bundling proposal modules with writeCoreProposal

For example: proposals/restart-vats-proposal.js starts with declarations such as import { M, mustMatch } from '@agoric/store';. To bundle it

1. invoke agoric run scripts/restart-vats.js
2. scripts/restart-vats.js has a default export that calls writeCoreProposal.

The links should go to the ta/deployment-types branch from PR #7895 to show static types. I might have missed in some cases.

The general approach is:

1. Write a module to carry out the proposal; feel free to import other modules as appropriate. This module consists of
   • one or more BootBehavior functions that take a powers argument.
   • one function to get a BootstrapManifest that refers to all the BootBehaviors, along with a BootstrapManifestPermit for each.
2. Write a script with 1 or more ProposalBuilders that refer to the function from the proposal to get a manifest. This module exports a DeployScriptFunction to call writeCoreProposal for each of the ProposalBuilders.
3. Run agoric run script-from-2.js to generate a bundle and a permit/script pair for each call to writeCoreProposal. Each of the scripts is generated code that executes the BootBehavior functions from the root module of the bundle.
4. Install the bundles using agd tx swingset install-bundle ... or the like
5. Use agd tx gov swingset core-eval ... or the like to make a governance proposal
6. stakeholders vote
7. If the proposal passes, the script is evaluated using the powers from the permit

The example above is packaged in a test that uses a yarn script to do the agoric run scripts/restart-vats.js part:

agoric-sdk/packages/vats/test/bootstrapTests/test-vats-restart.js

Line 90 in 8bcf6af

test.serial('run restart-vats proposal', async t => {

The test also shows that each resulting bundle has to be made available to the swingset kernel. The test short-circuits using a transaction to install the bundles, using controller.validateAndInstallBundle() directly. The EV(coreEvalBridgeHandler).fromBridge(bridgeMessage) part of the test simulates carrying out the proposal. The bridge message connects to the compartment.evaluate(code) above by way of bridgeCoreEval.

ref: agoric run

It seems to be missing from Agoric CLI Commands | Documentation

$ agoric run --help
Usage: agoric run [options] <script> [script-args...]

run a script with all your user privileges and some Agoric endowments

Options:
  -h, --help  display help for command

[dckc]

TODO: more details to explain

• getHelpers
• use in config files - entrypoint, module, args
• threading args
• examples from inter-protocol
• example in pegasus
• note that "bootstrap" and "core" are somewhat interchangeable: closely held capabilities that can affect the whole chain
• detail the globals available to a core eval script

[dckc]

another detail to explain: function in the manifest are looked up by name in the module namespace, so they have to be exported.

case in point:
Jorge-Lopes#17 (comment)

see office hours feb 7

[0xpatrickdev]

Cross-posting some findings from Discord:

Deploying Bundles

1. Ensure the b1-....json file is in your current file directory when deploying. It seems like the agd daemon might have issues resolving files in different directories.
2. Ensure to include an @ sign before the filename @b1-....json . Without this, you may see INSTALL_BUNDLE warn: SyntaxError#3: Unexpected token b in JSON at position 0
3. Include -b block with any agd tx command for an easy way to see if your rpc client accepts the transaction

# @see https://github.com/0xpatrickdev/agoric-vault-collateral-proposal/#1-deploy-bundles
NODE=http://localhost:26657
WALLET=dev-local
CHAIN_ID=agoriclocal
B1=b1-903e41a7c448a41b456298404a1c32c69302574209c6a5228723ed19e2dd99f2a693641196445bc27a90e19e1dfadfe6b3d9c9a93f080ffa33a70908e5af4fff.json
B2=b1-80e6fe68b299c82c2d26802c312bc37966a559f7b28f87d058887a79a9db48ad97da2240e71e3f98986071da8fc3c5d02358bec577b17a89cee2b1cb3cd23958.json


agd tx swingset install-bundle @$B1 --node $NODE --from $WALLET --chain-id $CHAIN_ID --gas=auto --gas-adjustment=1.2 -y -b block
agd tx swingset install-bundle @$B2 --node $NODE --from $WALLET --chain-id $CHAIN_ID --gas=auto --gas-adjustment=1.2 -y -b block

# depending on the chain settings, a `--fees=45000ubld` flag may also be needed

[kriskowal]

Also, use agoric follow :bundles in a side terminal. This will tell you when the bundles get accepted or rejected by the chain, and the reason why if not. agoric publish "$ENTRY" ... is intended to be a drop-in replacement for agd tx swingset install-bundle @"$ENTRY" …. Watch this space.

[dckc]

more on writeCoreProposal and such from #9093 by @Chris-Hibbert (cc @turadg ) ...

---

Proposal

The proposal is called with (powers, options) available. The manifest
detailing the powers that will be used is usually in the same file, and
conventionally named getManifestForFoo. The manifest needs to have a unique
name, since it will be referenced by name from the script. The usual format is

export const foo = async (
  {
    consume: {
      ...
    },
    brands: {
      ...
    }
  },
 options,
) => {
  // do the things using powers and options
};

export const getManifestForFoo = (powers, options) => {
  manifest: {
     [foo.name]: {
       consume: {
         ...
      },
  options,
)};

Script

The script describes how to build the core proposal for a3p-integration. For
agoric-3-proposals and uploading to the chain, the script is named in
CoreProposalSteps in app.go, and its defaultProposalBuilder is invoked
directly. The submission directories in a3p-integration are generated by the
generate-a3p-submissions.sh script, which reads instructions from
agoricProposal.sdk-generate in package.json. That field contains a list of
strings, each of which describes a single submission. If there is only one
submission, it can use the default directory name submission by specifying
only the name of the script file in builders/scripts/vars:

"sdk-generate": ["test-localchain"],

If there are multiple submissions, each entry has to specify the script name and
a distinct directory name.

"sdk-generate": [
  "probe-zcf-bundle probe-submission",
  "updatePriceFeeds priceFeed-submission",
  "add-auction newAuction-submission"
],

Script files should export defaultProposalBuilder and a default function
that invokes writeCoreProposal one or more times to generate sets of files
describing the proposal.

export const defaultProposalBuilder = ({ publishRef, install }) => {
  return harden({
    sourceSpec: '@agoric/vats/src/proposals/foo.js',
    getManifestCall: [
      'getManifestForFoo',
      {
        fooRef: publishRef(install('@agoric/...')),
        ...otherParams,
      },
    ],
  });
};
`
export default async (homeP, endowments) => {
  const { writeCoreProposal } = await makeHelpers(homeP, endowments);
  await writeCoreProposal('proposalName', defaultProposalBuilder);
};

The first element of getManifestCall is interpreted as the name of a proposal.
fooRef (i.e. publishRef(install(<path>))) is built from sources under
../packages, and passed as a bundleRef, which contains a bundleID suitable
for passing to Zoe or vatAdminService.

The second element of getManifestCall produces the options argument passed
to the proposal. (foo in the example above).``

---

agoric-sdk/a3p-integration/README.md

Lines 133 to 219 in 4c47805

	### Proposal
	The proposal is called with `(powers, options)` available. The manifest
	detailing the powers that will be used is usually in the same file, and
	conventionally named `getManifestForFoo`. The manifest needs to have a unique
	name, since it will be referenced by name from the script. The usual format is
	```js
	export const foo = async (
	{
	consume: {
	...
	},
	brands: {
	...
	}
	},
	options,
	) => {
	// do the things using powers and options
	};
	export const getManifestForFoo = (powers, options) => {
	manifest: {
	[foo.name]: {
	consume: {
	...
	},
	options,
	)};
	```
	### Script
	The script describes how to build the core proposal for `a3p-integration`. For
	`agoric-3-proposals` and uploading to the chain, the script is named in
	`CoreProposalSteps` in `app.go`, and its `defaultProposalBuilder` is invoked
	directly. The submission directories in a3p-integration are generated by the
	`generate-a3p-submissions.sh` script, which reads instructions from
	`agoricProposal.sdk-generate` in `package.json`. That field contains a list of
	strings, each of which describes a single submission. If there is only one
	submission, it can use the default directory name `submission` by specifying
	only the name of the script file in `builders/scripts/vars`:
	```json
	"sdk-generate": ["test-localchain"],
	```
	If there are multiple submissions, each entry has to specify the script name and
	a distinct directory name.
	```json
	"sdk-generate": [
	"probe-zcf-bundle probe-submission",
	"updatePriceFeeds priceFeed-submission",
	"add-auction newAuction-submission"
	],
	```
	Script files should export `defaultProposalBuilder` and a `default` function
	that invokes `writeCoreProposal` one or more times to generate sets of files
	describing the proposal.
	```js
	export const defaultProposalBuilder = ({ publishRef, install }) => {
	return harden({
	sourceSpec: '@agoric/vats/src/proposals/foo.js',
	getManifestCall: [
	'getManifestForFoo',
	{
	fooRef: publishRef(install('@agoric/...')),
	...otherParams,
	},
	],
	});
	};
	`
	export default async (homeP, endowments) => {
	const { writeCoreProposal } = await makeHelpers(homeP, endowments);
	await writeCoreProposal('proposalName', defaultProposalBuilder);
	};
	```
	The first element of `getManifestCall` is interpreted as the name of a proposal.
	`fooRef` (i.e. `publishRef(install(<path>))`) is built from sources under
	`../packages`, and passed as a `bundleRef`, which contains a `bundleID` suitable
	for passing to Zoe or `vatAdminService`.
	The second element of `getManifestCall` produces the `options` argument passed
	to the proposal. (`foo` in the example above).``