Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rebase/cherry-pick commits for upgrade 13 #8623

Merged
merged 42 commits into from
Dec 7, 2023

Conversation

mhofman
Copy link
Member

@mhofman mhofman commented Dec 6, 2023

This is a PR staging rebases of some master branch PRs towards an upgrade-13.

The following issue and associated PRs have been cherry picked:

These changes don't have any modifications to runtime files included in the vat bundles.

It was performed with the following interactive rebase onto the dev-upgrade-13 branch, and some minor conflict resolutions:

# Branch Agoric-ta-refactor-clientSupport
label base-Agoric-ta-refactor-clientSupport
pick 5a6601890 refactor: OfferMaker takes agoricNames
pick 175c892b8 refactor(clientSupport): move makeParseAmount
pick d5efec3c9 refactor(clientSupport): inline parseAmount
pick b6145e335 chore!: restrict exports from 'agoric' package
pick 6c50a2201 chore(deps): remove dep on 'agoric'
# pick 38919fcef ci: tighten cycles limit
# pick 8968b70f2 build: gitignore graph script output
pick d08967510 build: export for loadgen
label Agoric-ta-refactor-clientSupport
reset base-Agoric-ta-refactor-clientSupport
merge -C 7632299c0 Agoric-ta-refactor-clientSupport # Merge pull request #7824 from Agoric/ta/refactor-clientSupport

# Pull Request #8258
pick 2ac22d335 fix(inter-protocol): brands in CLI need not start with A-Z (#8258)

# Branch Agoric-8223-ibc-4
label base-Agoric-8223-ibc-4
pick 0ec1b79ea feat: update ibc-go to v4, adapt packages and API
pick 2b23487b4 build: eliminate dependence on gaia/ag0
pick a4fd51067 fix: handle hang-on-halt behavior from agoric-labs/cosmos-sdk#305
pick 4d998883c Revert "fix: handle hang-on-halt behavior from agoric-labs/cosmos-sdk#305"
pick 75d6b1dcc fix: update dependencies to fix tests
pick f0b6f78a3 chore: upgrade ibc-go to latest patch version
pick 49d28f7d8 docs: update changelog
label Agoric-8223-ibc-4
reset base-Agoric-8223-ibc-4
merge -C 28931a75d Agoric-8223-ibc-4 # Merge pull request #8409 from Agoric/8223-ibc-4

# Pull Request #8533
pick e26232e68 fix(solo): properly expose `/wallet/bridge.html` (#8533)

# Branch Agoric-mhofman-8535-fix-cli-vstorage-not-found
label base-Agoric-mhofman-8535-fix-cli-vstorage-not-found
pick 8c57bce80 fix(cli): handle not found error in vstorage requests
pick 21db80e57 chore(x/vstorage): add querier test
label Agoric-mhofman-8535-fix-cli-vstorage-not-found
reset base-Agoric-mhofman-8535-fix-cli-vstorage-not-found
merge -C c14f1bb06 Agoric-mhofman-8535-fix-cli-vstorage-not-found # Merge pull request #8538 from Agoric/mhofman/8535-fix-cli-vstorage-not-found

# Branch Agoric-sam-casting-capdata-errors
label base-Agoric-sam-casting-capdata-errors
pick 8f4ba3a2f fix(casting): dont crash on bad capdata
label Agoric-sam-casting-capdata-errors
reset base-Agoric-sam-casting-capdata-errors
merge -C 0001c4901 Agoric-sam-casting-capdata-errors # Merge pull request #8562 from Agoric/sam/casting-capdata-errors

# Branch Agoric-dc-setPrice-mcs
label base-Agoric-dc-setPrice-mcs
pick 9bf2d7293 fix(agops): fix continuing id lookup in oracle setPrice
label Agoric-dc-setPrice-mcs
reset base-Agoric-dc-setPrice-mcs
merge -C 3238732d5 Agoric-dc-setPrice-mcs # Merge pull request #8457 from Agoric/dc-setPrice-mcs

# Branch Agoric-mhofman-remove-test-bootstrap-upgrade
label base-Agoric-mhofman-remove-test-bootstrap-upgrade
pick d457bb78f chore(upgrade-test): various fixes
pick bd840f4e2 chore(upgrade-test): remove BOOTSTRAP_MODE
label Agoric-mhofman-remove-test-bootstrap-upgrade
reset base-Agoric-mhofman-remove-test-bootstrap-upgrade
merge -C 51399d19e Agoric-mhofman-remove-test-bootstrap-upgrade # Merge pull request #8598 from Agoric/mhofman/remove-test-bootstrap-upgrade

# Branch Agoric-mfig-follow-empty
label base-Agoric-mfig-follow-empty
pick 92a19285c fix(agoric): print errors while following
pick 97be818c4 fix(casting): properly follow an unpopulated state entry
label Agoric-mfig-follow-empty
reset base-Agoric-mfig-follow-empty
merge -C 8a11dec46 Agoric-mfig-follow-empty # Merge pull request #8604 from Agoric/mfig-follow-empty

# Branch Agoric-warner-8523-export-without-writelock
label base-Agoric-warner-8523-export-without-writelock
pick eb564f963 feat: add exporter.getHostKV() API
pick 3bc37990f fix: export state-sync snapshot without a DB write-lock
label Agoric-warner-8523-export-without-writelock
reset base-Agoric-warner-8523-export-without-writelock
merge -C 414e88b97 Agoric-warner-8523-export-without-writelock # Merge pull request #8619 from Agoric/warner/8523-export-without-writelock

# Branch Agoric-8570-return-grants
label base-Agoric-8570-return-grants
pick a88eb8a21 feat: pick up return-grants from latest cosmos-sdk
label Agoric-8570-return-grants
reset base-Agoric-8570-return-grants
merge -C 7728545e0 Agoric-8570-return-grants # Merge pull request #8616 from Agoric/8570-return-grants

# Branch Agoric-mhofman-8558-auto-provision
label base-Agoric-mhofman-8558-auto-provision
pick 75bd9a7bf feat(x/swingset): refuse smart wallet messages if not provisioned
# pick c0c7cc4a4 chore: Add comment back to cosmos from startWalletFactory
pick 30924ff97 chore(cosmos): always update swingset params on upgrade
pick 20a5485b9 feat(x/swingset): auto-provision smart wallet
pick d9cee0a0f chore(upgrade-test): add test of auto-provision
label Agoric-mhofman-8558-auto-provision
reset base-Agoric-mhofman-8558-auto-provision
merge -C dca42b9ff Agoric-mhofman-8558-auto-provision # Merge pull request #8602 from Agoric/mhofman/8558-auto-provision

# Pull Request #8582
pick 7153535c5 feat(cosmos): un-wire x/crisis (#8582)

# Manually authored
pick 457beb310 chore(web-components): fix lint

For reference, the non-pruned rebase-todo of the master commits since the one generated for upgrade-12 (#8510):

rebase-todo since upgrade-12
# Branch Agoric-ta-liveslots-types
label base-Agoric-ta-liveslots-types
pick d03ebfe57 chore(types): watchPromise
pick b3bc122f7 test: regression of VirtualObjectManager type
pick cc5c67919 chore(types): fix VirtualObjectManager
pick 5981fc881 chore(types): types for define* in vom
pick f7ac76cf7 chore(types): improve KindFacet
label Agoric-ta-liveslots-types
reset base-Agoric-ta-liveslots-types
merge -C 5dc325b8c Agoric-ta-liveslots-types # Merge pull request #8519 from Agoric/ta/liveslots-types

# Branch Agoric-8223-ibc-4
label base-Agoric-8223-ibc-4
pick 0ec1b79ea feat: update ibc-go to v4, adapt packages and API
pick 2b23487b4 build: eliminate dependence on gaia/ag0
pick a4fd51067 fix: handle hang-on-halt behavior from agoric-labs/cosmos-sdk#305
pick 4d998883c Revert "fix: handle hang-on-halt behavior from agoric-labs/cosmos-sdk#305"
pick 75d6b1dcc fix: update dependencies to fix tests
pick f0b6f78a3 chore: upgrade ibc-go to latest patch version
pick 49d28f7d8 docs: update changelog
label Agoric-8223-ibc-4
reset base-Agoric-8223-ibc-4
merge -C 28931a75d Agoric-8223-ibc-4 # Merge pull request #8409 from Agoric/8223-ibc-4

# Pull Request #8533
pick e26232e68 fix(solo): properly expose `/wallet/bridge.html` (#8533)

# Branch Agoric-mhofman-8535-fix-cli-vstorage-not-found
label base-Agoric-mhofman-8535-fix-cli-vstorage-not-found
pick 8c57bce80 fix(cli): handle not found error in vstorage requests
pick 21db80e57 chore(x/vstorage): add querier test
label Agoric-mhofman-8535-fix-cli-vstorage-not-found
reset base-Agoric-mhofman-8535-fix-cli-vstorage-not-found
merge -C c14f1bb06 Agoric-mhofman-8535-fix-cli-vstorage-not-found # Merge pull request #8538 from Agoric/mhofman/8535-fix-cli-vstorage-not-found

# Pull Request #8526
pick 0f42fb639 fix: ensure that we wrap pegasusConnectionsAdmin with E (#8526)

# Branch Agoric-refactor-bootstrap-test-support
label base-Agoric-refactor-bootstrap-test-support
pick 699d9e157 refactor: move benchmarkerator and test support code into their own packages
pick 8240485cb chore: refactor based on review comments
pick 3f9f8badf feat: add slog file output to benchmark tool
label Agoric-refactor-bootstrap-test-support
reset base-Agoric-refactor-bootstrap-test-support
merge -C 6d9ae67c5 Agoric-refactor-bootstrap-test-support # Merge pull request #8421 from Agoric/refactor-bootstrap-test-support

# Branch Agoric-sam-casting-capdata-errors
label base-Agoric-sam-casting-capdata-errors
pick 8f4ba3a2f fix(casting): dont crash on bad capdata
label Agoric-sam-casting-capdata-errors
reset base-Agoric-sam-casting-capdata-errors
merge -C 0001c4901 Agoric-sam-casting-capdata-errors # Merge pull request #8562 from Agoric/sam/casting-capdata-errors

# Branch Agoric-ta-privateArgs-types
label base-Agoric-ta-privateArgs-types
pick ecfcd96ad chore(types): fix typo
pick 3e5255daa chore(types): privateArgs for AdminFacet
label Agoric-ta-privateArgs-types
reset base-Agoric-ta-privateArgs-types
merge -C 275a1bd48 Agoric-ta-privateArgs-types # Merge pull request #8568 from Agoric/ta/privateArgs-types

# Branch Agoric-ta-typedoc
label base-Agoric-ta-typedoc
pick 6178a6d12 build: build-ts config
pick 3368e9e07 chore(types): misc fixes to build
pick 81c63492f refactor: re-export objectMap from Endo
pick 37e985510 build: generate docs
pick 5a82fb5b3 docs: demo annotations
pick cf66d243f chore(types): misc fixes for tsdoc parsing
pick d8cef7a56 ci: test building docs
label Agoric-ta-typedoc
reset base-Agoric-ta-typedoc
merge -C a0953fe7d Agoric-ta-typedoc # Merge pull request #8343 from Agoric/ta/typedoc

# Branch Agoric-typescript-5-3
label base-Agoric-typescript-5-3
pick 6104ed7ec chore(types): upgrade TypeScript to 5.3
label Agoric-typescript-5-3
reset base-Agoric-typescript-5-3
merge -C 76118c5ce Agoric-typescript-5-3 # Merge pull request #8571 from Agoric/typescript-5.3

# Branch Agoric-dc-setPrice-mcs
label base-Agoric-dc-setPrice-mcs
pick 9bf2d7293 fix(agops): fix continuing id lookup in oracle setPrice
label Agoric-dc-setPrice-mcs
reset base-Agoric-dc-setPrice-mcs
merge -C 3238732d5 Agoric-dc-setPrice-mcs # Merge pull request #8457 from Agoric/dc-setPrice-mcs

# Branch Agoric-mhofman-8578-upgrade-name-master
label base-Agoric-mhofman-8578-upgrade-name-master
pick 859f5bda1 chore: rename next unreleased upgrade
pick 65c6efa73 chore: remove upgrade-12 actions
pick 4a5710eeb docs(upgrade-test): revise readme
label Agoric-mhofman-8578-upgrade-name-master
reset base-Agoric-mhofman-8578-upgrade-name-master
merge -C 312271b6d Agoric-mhofman-8578-upgrade-name-master # Merge pull request #8587 from Agoric/mhofman/8578-upgrade-name-master

# Branch Agoric-ta-bump-better
label base-Agoric-ta-bump-better
pick 3cfe8396f chore(deps): bump better-sqlite to 9.1
label Agoric-ta-bump-better
reset base-Agoric-ta-bump-better
merge -C 4c174a21a Agoric-ta-bump-better # Merge pull request #8590 from Agoric/ta/bump-better

# Branch Agoric-gibson-2023-11-cleanup-coreproposalbehavior
label base-Agoric-gibson-2023-11-cleanup-coreproposalbehavior
pick a4364481e chore(deploy-script-support): Rename `allPowers` to clarify its limited scope
pick 4640c8ac0 chore(deploy-script-support): Move default restoreRef creation up a level
pick 520ddf836 chore(deploy-script-support): Clarify the [methodName, ...args] structure of getManifestCall
pick 037f890cd chore(deploy-script-support): Enforce bundleRef shape
pick e6903b5da chore(deploy-script-support): Rename `manifestNS` to clarify its nature
pick f16e093c4 chore(deploy-script-support): Clarify `shallowlyFulfilled`
pick cbef1af17 chore(deploy-script-support): Improve comment
pick f4abd3ad1 chore(deploy-script-support): Clean up logging
pick 6dc6f12db chore(deploy-script-support): Comment on the scope of `powers`
pick a3a54f971 chore(deploy-script-support): Rename variables for clarity
label Agoric-gibson-2023-11-cleanup-coreproposalbehavior
reset base-Agoric-gibson-2023-11-cleanup-coreproposalbehavior
merge -C 8406d71b7 Agoric-gibson-2023-11-cleanup-coreproposalbehavior # Merge pull request #8528 from Agoric/gibson-2023-11-cleanup-coreproposalbehavior

# Branch Agoric-mhofman-remove-test-bootstrap-upgrade
label base-Agoric-mhofman-remove-test-bootstrap-upgrade
pick d457bb78f chore(upgrade-test): various fixes
pick bd840f4e2 chore(upgrade-test): remove BOOTSTRAP_MODE
label Agoric-mhofman-remove-test-bootstrap-upgrade
reset base-Agoric-mhofman-remove-test-bootstrap-upgrade
merge -C 51399d19e Agoric-mhofman-remove-test-bootstrap-upgrade # Merge pull request #8598 from Agoric/mhofman/remove-test-bootstrap-upgrade

# Branch Agoric-mfig-follow-empty
label base-Agoric-mfig-follow-empty
pick 92a19285c fix(agoric): print errors while following
pick 97be818c4 fix(casting): properly follow an unpopulated state entry
label Agoric-mfig-follow-empty
reset base-Agoric-mfig-follow-empty
merge -C 8a11dec46 Agoric-mfig-follow-empty # Merge pull request #8604 from Agoric/mfig-follow-empty

# Branch Agoric-ta-typecheck-scripts
label base-Agoric-ta-typecheck-scripts
pick 843bf7dde chore(types): type provisionPoolStartResult
pick 66a9fcc2b chore(types): checkJs all vats package
pick a0b701d92 chore(types): remove redundant ts-check
pick fbdb3353c chore(types): fuller watchPromise
pick 09a77a61a chore(types): PromiseWatcher
pick b65c9ee6e chore(deps): bump typescript-eslint to 6.13.2
pick 1d2278278 chore(types): providePromiseWatcher, new defaults
label Agoric-ta-typecheck-scripts
reset base-Agoric-ta-typecheck-scripts
merge -C 1b5e57f17 Agoric-ta-typecheck-scripts # Merge pull request #8596 from Agoric/ta/typecheck-scripts

# Branch Agoric-warner-8523-export-without-writelock
label base-Agoric-warner-8523-export-without-writelock
pick eb564f963 feat: add exporter.getHostKV() API
pick 3bc37990f fix: export state-sync snapshot without a DB write-lock
label Agoric-warner-8523-export-without-writelock
reset base-Agoric-warner-8523-export-without-writelock
merge -C 414e88b97 Agoric-warner-8523-export-without-writelock # Merge pull request #8619 from Agoric/warner/8523-export-without-writelock

# Branch Agoric-8455-fakeVirtual-watchedPromises
label base-Agoric-8455-fakeVirtual-watchedPromises
pick 8129d45b4 test: support for watchedPromises in fakeVirtualSupport
label Agoric-8455-fakeVirtual-watchedPromises
reset base-Agoric-8455-fakeVirtual-watchedPromises
merge -C 241c60a3d Agoric-8455-fakeVirtual-watchedPromises # Merge pull request #8618 from Agoric/8455-fakeVirtual-watchedPromises

# Branch Agoric-8570-return-grants
label base-Agoric-8570-return-grants
pick a88eb8a21 feat: pick up return-grants from latest cosmos-sdk
label Agoric-8570-return-grants
reset base-Agoric-8570-return-grants
merge -C 7728545e0 Agoric-8570-return-grants # Merge pull request #8616 from Agoric/8570-return-grants

# Branch Agoric-mhofman-8558-auto-provision
label base-Agoric-mhofman-8558-auto-provision
pick 75bd9a7bf feat(x/swingset): refuse smart wallet messages if not provisioned
pick c0c7cc4a4 chore: Add comment back to cosmos from startWalletFactory
pick 30924ff97 chore(cosmos): always update swingset params on upgrade
pick 20a5485b9 feat(x/swingset): auto-provision smart wallet
pick d9cee0a0f chore(upgrade-test): add test of auto-provision
label Agoric-mhofman-8558-auto-provision
reset base-Agoric-mhofman-8558-auto-provision
merge -C dca42b9ff Agoric-mhofman-8558-auto-provision # Merge pull request #8602 from Agoric/mhofman/8558-auto-provision

# Pull Request #8582
pick 7153535c5 feat(cosmos): un-wire x/crisis (#8582)

turadg and others added 25 commits December 6, 2023 16:58
it's only for installing the bin and importing that bin module
refactor clientSupport to not depend on agoric-cli package
* fix(inter-protocol): brands in CLI need not start with A-Z

* test(inter-protocol): clientSupport for initial non-uppercase

* fixup! fix(inter-protocol): brands in CLI need not start with A-Z

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
The backport referenced above has the agd process hang around after halting.
Wrote a wrapper utility which scans stderr for the halt application message,
interrupts the process, then propagates its error signal. Conventional shell
scripting is not up to the challenge of expressing this behavior.
…#305"

This reverts commit efcc3b0.

We decided to roll back the problematic change (agoric-labs/cosmos-sdk#305).
feat: update ibc-go to v4, adapt packages and API
…t-found

fix(cli): handle not found error in vstorage requests
fix(casting): dont crash on bad capdata
fix(agops): fix continuing id lookup in oracle setPrice
@mhofman mhofman requested a review from toliaqat December 6, 2023 17:42
@mhofman mhofman force-pushed the mhofman/cherry-pick-upgrade-13 branch from ef2163e to 0e894b8 Compare December 6, 2023 18:36
mergify bot and others added 15 commits December 6, 2023 20:06
…grade

Remove upgrade-test BOOTSTRAP_MODE=test
fix(casting): properly follow an unpopulated node
Add a new API to the exporter, `exporter.getHostKV(key)`, so that the
cosmic-swingset state-sync exporter process can query `host.height`
without accidentally creating a read-write transaction too.

refs #8523
Exporting a state-sync snapshot is a read-only operation, and is
designed to run "in the background", i.e. in parallel with normal
mutating operations. It accomplishes this by opening a read-only
transaction right away, effectively capturing a snapshot of the SQLite
database state, to insulate the export process from ongoing writes by
the execution host.

The cosmic-swingset exporter starts with a query of `host.height`, to
confirm that the database has not already advanced to a new block
before this snapshot/read-transaction can be taken.

Previously, this query worked by using `openSwingStore`, and then
calling `hostStorage.hostKVStore.get('host.height')`. This had two
problems:

* TOCTTOU: the `hostKVStore.get` used a different DB connection (and
  different txn) than the exporter, so it might return a different
  height, negating the accuracy of the consistency check

* read-write txn: `openSwingStore` creates a read-*write* txn, even
  when merely opening the DB (because it might need to create the
  initial tables). This txn is closed right away, before
  `openSwingStore()` returns, so it did not present a threat to
  ongoing operations. But if the exporter was created while the
  ongoing execution side already had its own read-write txn
  open (e.g. while `controller.run()` was running), then it would
  fail, and `makeSwingStoreExporter` would fail with `SQLITE_BUSY`

Instead, we take advantage of the new `swingStoreExporter.getHostKV()`
API, and use *it* to fetch `host.height`. Unlike the normal
swingstore, the swingstore-exporter refrains from creating read-write
transactions entirely. So the cosmic-swingset export code can safely
query the height without fear of getting the wrong value or failing
because of an ongoing write transaction.

We think this should fix the SQLITE_BUSY errors.

refs #8523
…lock

fix: export state-sync snapshot without a DB write-lock
feat: allow clawback vesting accounts to return grants
* feat(cosmos): un-wire x/crisis

* fixup! feat(cosmos): un-wire x/crisis

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
@mhofman mhofman force-pushed the mhofman/cherry-pick-upgrade-13 branch from 0e894b8 to 457beb3 Compare December 6, 2023 20:08
Copy link
Contributor

@toliaqat toliaqat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Look great! Thanks @mhofman for making it happen.
Verified that the PRs are ones referenced in the upgrade-13 project page.
Verified that these changes will not impact Vats code.
Replicated cherry-pick of the upgrade-13 project PRs and found no difference between my replication and this PR.
🚀

@mhofman mhofman added the force:integration Force integration tests to run on PR label Dec 7, 2023
@mhofman mhofman merged commit 24acd13 into dev-upgrade-13 Dec 7, 2023
64 checks passed
@mhofman mhofman deleted the mhofman/cherry-pick-upgrade-13 branch December 7, 2023 01:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
force:integration Force integration tests to run on PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants