Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(test): test opcode programs in different scenarios #808

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

winsvega
Copy link
Collaborator

@winsvega winsvega commented Sep 16, 2024

🗒️ Description

Conversion of opcode diff places tests by ori.
A test defines series of test scenarios that are run on each parametrized opcode sequence.
Then we check if the sequence worked as expected in a given scenario.

I think this is a powerful method to template test any new given opcode.
we already have pre defined scenarios. then we just add one more parameter with what we want to test, and it will be covered on all the cases automatically.

Cases can be like:
callcode->staticcall-> [opcode]
create2-> [opcode]
[opcode] -> revert

check it out. so we can define opcode programs and scenarios. then the test will put each opcode program in each scenario and verify that it's result is the same (perhaps result will be complex depending on context and fork)

the idea is so far to have a template test and then we can easily just add opcode programms and it will be run in all crazy combinations.
likce call delegate call suicide revert and so on

This is still WIP.

🔗 Related Issues

#184

✅ Checklist

  • All: Set appropriate labels for the changes.
  • All: Considered squashing commits to improve commit history.
  • All: Added an entry to CHANGELOG.md.
  • All: Considered updating the online docs in the ./docs/ directory.
  • Tests: All converted JSON/YML tests from ethereum/tests have been added to converted-ethereum-tests.txt.
  • Tests: A PR with removal of converted JSON/YML tests from ethereum/tests have been opened.
  • Tests: Included the type and version of evm t8n tool used to locally execute test cases: e.g., ref with commit hash or geth 1.13.1-stable-3f40e65.
  • Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.

Copy link
Member

@chfast chfast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you give it any better name? The "opcode diff places" has no meaning.

@winsvega
Copy link
Collaborator Author

it translates as opcode in different (logical) places

@winsvega winsvega closed this Sep 20, 2024
@winsvega winsvega reopened this Sep 20, 2024
@winsvega winsvega changed the title feat(test): opcode diff places test feat(test): test opcode programs in different scenarios Sep 20, 2024
Copy link
Member

@marioevz marioevz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few suggestions after a quick review. I haven't checked files in the ./scenarios/ scenarios/ folder, but will do on the re-review.

tests/frontier/scenarios/test_scenarios.py Outdated Show resolved Hide resolved
tests/frontier/scenarios/common.py Outdated Show resolved Hide resolved
tests/frontier/scenarios/common.py Show resolved Hide resolved
tests/frontier/scenarios/common.py Outdated Show resolved Hide resolved
@winsvega winsvega force-pushed the dailytest branch 2 times, most recently from b1af590 to ff2eeff Compare September 24, 2024 13:13
@winsvega winsvega force-pushed the dailytest branch 2 times, most recently from b28f2ef to f7de36c Compare October 17, 2024 10:02
@winsvega winsvega added scope:pytest Scope: Pytest plugins type:feat type: Feature labels Oct 28, 2024
@winsvega winsvega force-pushed the dailytest branch 11 times, most recently from 5d3dd84 to 940cb3e Compare October 31, 2024 14:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
scope:pytest Scope: Pytest plugins type:feat type: Feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants