Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Solidity tests JS benchmark #506

Merged
merged 4 commits into from
Jun 20, 2024

Conversation

agostbiro
Copy link
Member

@agostbiro agostbiro commented Jun 12, 2024

Adds a JS benchmark command to execute certain forge-std tests to benchmark the EDR Solidity tests implementation against forge. The benchmark can be executed with pnpm i && pnpm soltests from the crates/tools/js/benchmark directory. There is no difference in execution time on my machine between forge test and pnpm soltests.

Baseline:

git clone https://github.com/NomicFoundation/forge-std.git
cd forge-std
git checkout js-benchmark-config
foundryup --commit 0a5b22f07
forge test --no-match-contract 'StdChainsTest|StdCheatsTest|MockERC721Test|MockERC20Test|StdCheatsForkTest|StdJsonTest|StdUtilsForkTest|StdTomlTest'

The ignored forge-std tests use fork mode or depend on Foundry project format.

Copy link

changeset-bot bot commented Jun 12, 2024

⚠️ No Changeset found

Latest commit: 2d6fc16

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@agostbiro agostbiro temporarily deployed to github-action-benchmark June 12, 2024 16:30 — with GitHub Actions Inactive
@agostbiro agostbiro marked this pull request as draft June 12, 2024 16:30
@agostbiro agostbiro had a problem deploying to github-action-benchmark June 17, 2024 16:45 — with GitHub Actions Failure
@agostbiro agostbiro had a problem deploying to github-action-benchmark June 18, 2024 11:41 — with GitHub Actions Failure
@agostbiro agostbiro marked this pull request as ready for review June 18, 2024 11:41
@agostbiro agostbiro force-pushed the misc/js-solidity-tests-benchmark branch from 2ace0c3 to a3e5d3d Compare June 18, 2024 12:37
@agostbiro agostbiro had a problem deploying to github-action-benchmark June 18, 2024 12:37 — with GitHub Actions Failure
@agostbiro agostbiro force-pushed the misc/js-solidity-tests-benchmark branch from a3e5d3d to f9222f5 Compare June 18, 2024 15:57
@agostbiro agostbiro temporarily deployed to github-action-benchmark June 18, 2024 15:57 — with GitHub Actions Inactive
@agostbiro agostbiro added the no changeset needed This PR doesn't require a changeset label Jun 18, 2024
@agostbiro agostbiro self-assigned this Jun 18, 2024
@agostbiro agostbiro linked an issue Jun 18, 2024 that may be closed by this pull request
@agostbiro agostbiro requested review from fvictorio and Wodann June 18, 2024 16:10
@Wodann
Copy link
Member

Wodann commented Jun 18, 2024

There seem to be issues in CI that I'm not seeing on other PRs. Could a setting have affected the deserialisation logic?

@agostbiro agostbiro force-pushed the feat/solidity-tests branch 2 times, most recently from 6e7d97f to 8fb36f6 Compare June 19, 2024 10:06
@agostbiro agostbiro force-pushed the misc/js-solidity-tests-benchmark branch from f9222f5 to 3cc66bc Compare June 19, 2024 10:13
@agostbiro agostbiro temporarily deployed to github-action-benchmark June 19, 2024 10:13 — with GitHub Actions Inactive
@agostbiro
Copy link
Member Author

There seem to be issues in CI that I'm not seeing on other PRs. Could a setting have affected the deserialisation logic?

Thanks, it was a problem in the base branch after merging main into it. It should be fixed now.

@agostbiro agostbiro force-pushed the feat/solidity-tests branch from 8fb36f6 to 5f79eed Compare June 19, 2024 11:17
@agostbiro agostbiro force-pushed the misc/js-solidity-tests-benchmark branch from 3cc66bc to 2d6fc16 Compare June 19, 2024 11:19
@agostbiro agostbiro temporarily deployed to github-action-benchmark June 19, 2024 11:19 — with GitHub Actions Inactive
@agostbiro agostbiro merged commit e9228c7 into feat/solidity-tests Jun 20, 2024
32 checks passed
@agostbiro agostbiro deleted the misc/js-solidity-tests-benchmark branch June 20, 2024 05:58
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 19, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
no changeset needed This PR doesn't require a changeset
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create Solidity tests benchmark to understand JS overhead
2 participants