Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci(integration): add a step to archive core eval scripts #9929

Merged
merged 6 commits into from
Aug 22, 2024

Conversation

LuqiPan
Copy link
Contributor

@LuqiPan LuqiPan commented Aug 20, 2024

ad-hoc PR

Description

We'd like to archive core proposals as they're built in test-docker-build job

Security Considerations

No secrets should be archived by this step

Scaling Considerations

Documentation Considerations

Testing Considerations

Will run integration tests in this PR to test this change

Upgrade Considerations

@LuqiPan LuqiPan added the force:integration Force integration tests to run on PR label Aug 20, 2024
Copy link

cloudflare-workers-and-pages bot commented Aug 20, 2024

Deploying agoric-sdk with  Cloudflare Pages  Cloudflare Pages

Latest commit: 3664a86
Status: ✅  Deploy successful!
Preview URL: https://51cc9b80.agoric-sdk.pages.dev
Branch Preview URL: https://archive-core-proposals.agoric-sdk.pages.dev

View logs

@LuqiPan LuqiPan marked this pull request as ready for review August 21, 2024 19:08
@dckc
Copy link
Member

dckc commented Aug 21, 2024

Where should we expect to find the artifacts?

.github/workflows/integration.yml Show resolved Hide resolved
with:
name: core-proposals
path: ./*.zip
- name: remove all zip files after they're archived
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is required to pass the git tree dirty check later on in this workflow

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you create the zips in a path outside the repo? e.g. /tmp/proposal-zips

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or in the repo but under a .gitignored path. e.g. build. That should avoid the dirty check.

We shouldn't need another step for cleanup

.github/workflows/integration.yml Show resolved Hide resolved
@LuqiPan LuqiPan force-pushed the archive-core-proposals branch from c7d997c to 3b9b28f Compare August 21, 2024 19:10
@LuqiPan
Copy link
Contributor Author

LuqiPan commented Aug 21, 2024

Where should we expect to find the artifacts?

I'm planning to add following comments to the yaml file, is the explanation clear enough for you? An successful run from previous commit can be seen here: https://github.com/Agoric/agoric-sdk/actions/runs/10494325003

        # This zip file can be found at the bottom of `Summary` page of
        # `Integration tests` workflow once the workflow is completed.

.github/workflows/integration.yml Show resolved Hide resolved
uses: actions/upload-artifact@v4
with:
name: core-proposals
path: ./*.zip
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why zip instead of uploading the dirs?

    path: ./a3p-integration/proposals/*/submission

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because upload-artifact action doesn't like it when there's : in the path, we have b:upgrade-next in the path which would prevent the action from uploading. So I opted to zip while replacing the : with - instead of renaming the directories

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah. Please document that in a comment on the zipping step

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternately, it would be less code and perhaps a better debugging experience if instead of zipping the submissions were copied to a /tmp folder and this step uploaded from there. (copy with rename of parent path)

.github/workflows/integration.yml Outdated Show resolved Hide resolved
- collect core proposals after proposals tests
- collect core proposals under /tmp/core_proposals
- add comments on how to find the archived core proposals
@LuqiPan LuqiPan requested a review from turadg August 21, 2024 21:34
@LuqiPan
Copy link
Contributor Author

LuqiPan commented Aug 21, 2024

Thanks to suggestions by @turadg, the file structure for the artifact is cleaner too!

tree ~/Downloads/core-proposals/
/Users/luqi/Downloads/core-proposals/
├── a-vaults-auctions
│   ├── add-auction-permit.json
│   ├── add-auction-plan.json
│   ├── add-auction.js
│   ├── b1-8859b141114716b24cca1bd8bc14f81c066880556b5e94eb1767c0ca3d5f4917a6762dcbab85d84bcdf06ba64179a34bfd7cbb5b43c9ab459b5abe09aeb7cdd9.json
│   ├── b1-cdcbc0eabdca825a2ec45d054a6f9914770f3bc64b724d93fe3eef8422a0e729e4a7d8f201e120a833689aa54728e143a4289b80057d8fad102744d2f2bbaee0.json
│   ├── b1-edc85da00cd6d96c7a70d94120b1f40f6c5e2ba6a6b2c4b7d2e7a0b84467c271e75c1ee596e360e5b6382564d1638be1a5daa01d93864bfc96cd553028dd5ac1.json
│   ├── upgrade-vaults-permit.json
│   ├── upgrade-vaults-plan.json
│   └── upgrade-vaults.js
├── b-enable-orchestration
│   ├── b1-080b19b0b18b89b5723552355339b06705f1c47bb115c1fe7f0f167cbffec2ad1664ed1d0fdceb325f34ac58206f9cf45016d1723f3605eb40852e9b0600356f.json
│   ├── b1-58020955239bc4a30c5a0ebd18ad01c7d27e53346bec134e0e9d9c3280b8291ae951b3d416bdaffb7f1e5e6a65494b5a0140cc01b74aba4994ed4e028d2836aa.json
│   ├── b1-bfc0253e9c10b864eabdd0f782670434012839e5326238ba70b7d9b8df2fff30ff8ed41d0880b1e09c5f24caff68a55513d8ac3aea62e2fe3e62ce15c41495d3.json
│   ├── b1-ead8480a5d820b74d8144ed562c2a5e7de20e530608964a039e5c0946df401e14ca59a274a1d5a6abe3ec36526b798e417ffe57c092985207f90fb46805f1f7c.json
│   ├── gov-orchestration-permit.json
│   ├── gov-orchestration-plan.json
│   ├── gov-orchestration.js
│   ├── upgrade-wallet-factory-permit.json
│   ├── upgrade-wallet-factory-plan.json
│   └── upgrade-wallet-factory.js
└── c-stake-bld
    ├── b1-71b117d12f7a0d2238fcccfe655aeb719a73af5dffa8976390a0f7edb7a30e0951f449177a671e316508d36a6ca6a9a58e5c0592dac73ce914477e5661fbbe2a.json
    ├── b1-92b293b43b9b01e7f308e8b22b62669108cbe6381d3a3a60b19568f2f59a50016fe1476c270f47d2bb7a2bf05e44f332406bb3488745067036e8ca28fc30cf38.json
    ├── start-stakeBld-permit.json
    ├── start-stakeBld-plan.json
    └── start-stakeBld.js

@dckc
Copy link
Member

dckc commented Aug 21, 2024

Where should we expect to find the artifacts?

Thanks for the clues, @LuqiPan

We follow the test-docker-build Details link...

image

then back up to the Summary

image

and finally at the bottom, we see core-proposals that we can download:

image

.github/workflows/integration.yml Outdated Show resolved Hide resolved
.github/workflows/integration.yml Outdated Show resolved Hide resolved
Copy link
Member

@dckc dckc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is working to my satisfaction.

Looks like @turadg has some helpful suggestions on comments etc; for now, I'll stand by for that discussion to conclude before approving.

@LuqiPan LuqiPan requested a review from turadg August 21, 2024 22:21
Copy link
Member

@turadg turadg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Be sure to automerge:squash so keep the history clean in master

@LuqiPan LuqiPan added the automerge:squash Automatically squash merge label Aug 21, 2024
@LuqiPan LuqiPan changed the title ci(integration): add a step to archive core proposal artifacts ci(integration): add a step to archive core eval scripts Aug 21, 2024
@mergify mergify bot merged commit 03f13e7 into master Aug 22, 2024
80 checks passed
@mergify mergify bot deleted the archive-core-proposals branch August 22, 2024 00:20
kriskowal pushed a commit that referenced this pull request Aug 27, 2024
ad-hoc PR

## Description


We'd like to archive core proposals as they're built in test-docker-build job

### Security Considerations


No secrets should be archived by this step

### Scaling Considerations


### Documentation Considerations


### Testing Considerations


Will run integration tests in this PR to test this change

### Upgrade Considerations
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
automerge:squash Automatically squash merge force:integration Force integration tests to run on PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants