-
Notifications
You must be signed in to change notification settings - Fork 91
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fuzzing tests #925
Comments
I thought there was a reason why we didn't implement echidna when I asked about it |
At that time we didn't have a clear scope of what to fuzz. Now we have the LibUbiquityPool which is going to be deployed on mainnet, so the contract scope is clear. |
I think that auditors from our 1st Sherlock audit have already performed all of the fuzz, invariant and formal verification tests so it makes sense to set a normal priority for: |
Good point, I remember that quite a few issues were detected with different fuzzers on top of manual code reviews. |
@gitcoindev the deadline is at 2024-05-09T10:02:25.473Z |
/start |
! Too many assigned issues, you have reached your max limit |
I will work on this behind the scenes and re-start the bot when the other issues are closed. |
@gitcoindev do you mind if i work on this one,if you have other tasks? |
Hi @3scava1i3r I have already started (the other tasks are finished, just waiting to be officially closed), but there is good news for you. We have a twin task for invariant tests here: #563 , would you mind picking that one instead? |
@3scava1i3r please confirm, whether you would be fine working on the twin task mentioned in the comment above - invariant test. If you are willing to work on this, please execute start command in #563 and the task will be automatically assigned to you. |
Hey there! @gitcoindev 👋 I've been working on the 'Better Security' tasks and wanted to ask for some assistance. I tried using the 'start' command to pick a task, but it doesn't seem to be working for me. I've made some progress and don't want to interrupt the workflow or cause any issues. Could you please guide me on how |
No sir,I will find something else to work on then |
Hi @0xJoichiro . Let's try to fix picking the task for you. There are a few commands available in the workflow. If you type |
/help |
Available Commands
|
/help |
# Skipping to list available commands. |
understood currently working on #927 using openzeplin defender |
fuzzing testing upcoming? @gitcoindev |
Yes, I will open a pull request until Friday EOD. |
I (no AI involved) identified the following properties of the system in case of mint, redeem and collection redemption : Dollar token mint properties:
Dollar token redeem properties:
Collecting redemption properties:
Those properties will be a base for the fuzz tests. Multiple inputs shall be provided by the test harness to test each of the properties. |
Sounds good. The same "properties" could be later converted to invariants. |
/start |
Tips:
|
Draft pull request with a skeleton for tests opened. I will mark it as ready for review when all the fuzz tests based on properties above will be implemented. |
Pull request ready for the review. |
/start |
! Too many assigned issues, you have reached your max limit |
Interesting, I received a notification in the pull request that bot unassigned me, so I tried to re-assign myself. |
+ Evaluating results. Please wait... |
|
View | Contribution | Count | Reward |
---|---|---|---|
Issue | Comment | 1 | 0.4 |
Review | Comment | 1 | 0.558 |
Conversation Incentives
Comment | Formatting | Relevance | Reward |
---|---|---|---|
I thought there was a reason why we didn't implement echidna whe… | 1.6content: p: count: 16 score: 1 wordValue: 0.1 formattingMultiplier: 1 | 0.25 | 0.4 |
2% seems steep? Why was this decided to be the default? Please t… | 3.1content: p: count: 31 score: 1 wordValue: 0.1 formattingMultiplier: 1 | 0.18 | 0.558 |
[ 37.5385 WXDAI ]
@rndquu
Contributions Overview
View | Contribution | Count | Reward |
---|---|---|---|
Issue | Specification | 1 | 16.2 |
Issue | Comment | 3 | 10.273 |
Review | Comment | 4 | 11.0655 |
Conversation Incentives
Comment | Formatting | Relevance | Reward |
---|---|---|---|
We should implement fuzzing tests for [LibUbiquityPool](https://… | 16.2content: p: count: 151 score: 1 a: count: 9 score: 1 code: count: 2 score: 1 wordValue: 0.1 formattingMultiplier: 1 | 1 | 16.2 |
At that time we didn't have a clear scope of what to fuzz. Now w… | 6.6content: p: count: 32 score: 1 a: count: 1 score: 1 wordValue: 0.2 formattingMultiplier: 1 | 0.845 | 5.577 |
I think that auditors from our 1st Sherlock audit have already p… | 7.4content: p: count: 37 score: 1 wordValue: 0.2 formattingMultiplier: 1 | 0.44 | 3.256 |
Sounds good. The same "properties" could be later converted to [… | 2.4content: p: count: 11 score: 1 a: count: 1 score: 1 wordValue: 0.2 formattingMultiplier: 1 | 0.6 | 1.44 |
@0x4007 That 2% fee is used solely in the test files. Mint and … | 2.6content: p: count: 25 score: 1 a: count: 1 score: 1 wordValue: 0.1 formattingMultiplier: 1 | 0.31 | 0.806 |
@gitcoindev How did you come up with the `90000` number?… | 3.1content: p: count: 28 score: 1 code: count: 2 score: 1 a: count: 1 score: 1 wordValue: 0.1 formattingMultiplier: 1 | 0.85 | 2.635 |
This seems to be a low severity issue but keeping in mind that n… | 3.2content: p: count: 31 score: 1 code: count: 1 score: 1 wordValue: 0.1 formattingMultiplier: 1 | 0.81 | 2.592 |
@gitcoindev 1. Could you add 2 more fuzz tests: a) If user mint… | 6.1content: p: count: 61 score: 1 wordValue: 0.1 formattingMultiplier: 1 | 0.825 | 5.0325 |
[ 390.696 WXDAI ]
@gitcoindev
Contributions Overview
View | Contribution | Count | Reward |
---|---|---|---|
Issue | Task | 1 | 300 |
Issue | Comment | 10 | 0 |
Review | Comment | 10 | 90.696 |
Conversation Incentives
Comment | Formatting | Relevance | Reward |
---|---|---|---|
Good point, I remember that quite a few issues were detected wit… | 0content: p: count: 20 score: 1 wordValue: 0 formattingMultiplier: 0 | 0.57 | - |
I will work on this behind the scenes and re-start the bot when … | 0content: p: count: 18 score: 1 wordValue: 0 formattingMultiplier: 0 | 0.36 | - |
Hi @3scava1i3r I have already started (the other tasks are finis… | 0content: p: count: 42 score: 1 wordValue: 0 formattingMultiplier: 0 | 0.35 | - |
@3scava1i3r please confirm, whether you would be fine working on… | 0content: p: count: 44 score: 1 wordValue: 0 formattingMultiplier: 0 | 0.22 | - |
Hi @0xJoichiro . Let's try to fix picking the task for you. Ther… | 0content: p: count: 143 score: 1 code: count: 3 score: 1 wordValue: 0 formattingMultiplier: 0 | 0.68 | - |
Yes, I will open a pull request until Friday EOD. | 0content: p: count: 10 score: 1 wordValue: 0 formattingMultiplier: 0 | 0.76 | - |
I (no AI involved) identified the following properties of the sy… | 0content: p: count: 408 score: 1 code: count: 14 score: 1 wordValue: 0 formattingMultiplier: 0 | 0.725 | - |
Draft pull request with a skeleton for tests opened. I will mark… | 0content: p: count: 29 score: 1 wordValue: 0 formattingMultiplier: 0 | 0.69 | - |
Pull request ready for the review. | 0content: p: count: 6 score: 1 wordValue: 0 formattingMultiplier: 0 | 0.715 | - |
Interesting, I received a notification in the pull request that … | 0content: p: count: 19 score: 1 wordValue: 0 formattingMultiplier: 0 | 0.73 | - |
Resolves https://github.com/ubiquity/ubiquity-dollar/issues/925 … | 0content: p: count: 7 score: 1 wordValue: 0 formattingMultiplier: 0 | 0.71 | - |
@0x4007 yes, as @rndquu mentioned, on deployment fees are set to… | 8.4content: p: count: 21 score: 1 wordValue: 0.2 formattingMultiplier: 2 | 0.6 | 5.04 |
@rndquu when I executed longer fuzzing sessions I frequently enc… | 32.4content: p: count: 81 score: 1 wordValue: 0.2 formattingMultiplier: 2 | 0.67 | 21.708 |
This pull request is still a draft. It contains a skeleton for f… | 12content: p: count: 30 score: 1 wordValue: 0.2 formattingMultiplier: 2 | 0.62 | 7.44 |
I am playing with the fuzzer a lot and discovered an interesting… | 42content: p: count: 92 score: 1 code: count: 13 score: 1 wordValue: 0.2 formattingMultiplier: 2 | 0.65 | 27.3 |
I will finish with the whole test suite till Friday EOD and set … | 7.6content: p: count: 19 score: 1 wordValue: 0.2 formattingMultiplier: 2 | 0.59 | 4.484 |
I pushed the remaining fuzz tests also for redemption delay, set… | 6.4content: p: count: 16 score: 1 wordValue: 0.2 formattingMultiplier: 2 | 0.625 | 4 |
Sure, thank you for the feedback. | 2.4content: p: count: 6 score: 1 wordValue: 0.2 formattingMultiplier: 2 | 0.66 | 1.584 |
Hi @rndquu @0x4007 @molecula451 the PR is ready to review, I exe… | 22.8content: p: count: 57 score: 1 wordValue: 0.2 formattingMultiplier: 2 | 0.65 | 14.82 |
@molecula451 I moved fuzz tests as requested, could you please h… | 6.4content: p: count: 16 score: 1 wordValue: 0.2 formattingMultiplier: 2 | 0.675 | 4.32 |
[ 0.1225 WXDAI ]
@3scava1i3r
Contributions Overview
View | Contribution | Count | Reward |
---|---|---|---|
Issue | Comment | 2 | 0.1225 |
Conversation Incentives
Comment | Formatting | Relevance | Reward |
---|---|---|---|
@gitcoindev do you mind if i work on this one,if you have other … | 0.35content: p: count: 14 score: 1 wordValue: 0.1 formattingMultiplier: 0.25 | 0.2 | 0.07 |
No sir,I will find something else to work on then | 0.25content: p: count: 10 score: 1 wordValue: 0.1 formattingMultiplier: 0.25 | 0.21 | 0.0525 |
[ 0.0445 WXDAI ]
@0xJoichiro
Contributions Overview
View | Contribution | Count | Reward |
---|---|---|---|
Issue | Comment | 2 | 0.0445 |
Conversation Incentives
Comment | Formatting | Relevance | Reward |
---|---|---|---|
Hey there! @gitcoindev 👋 I've been working on the 'Better Secu… | 2.225content: p: count: 89 score: 1 wordValue: 0.1 formattingMultiplier: 0.25 | 0.02 | 0.0445 |
understood currently working on https://github.com/ubiquity/ubiq… | 0.2content: p: count: 8 score: 1 wordValue: 0.1 formattingMultiplier: 0.25 | - | - |
[ 0 WXDAI ]
@molecula451
Contributions Overview
View | Contribution | Count | Reward |
---|
Conversation Incentives
Comment | Formatting | Relevance | Reward |
---|
We should implement fuzzing tests for LibUbiquityPool. I think it would be enough to fuzz only "user scenarios":
Possible solutions for fuzzing tests (I would simply start with foundry since we're using it as a development framework + it also contains invariant testing):
What should be done:
UbiquityPoolFacet.fuzz.t.sol
so we could distinguish unit and fuzz tests)a) On PR fuzzing tests should run with a small number of runs (so that we could merge PRs fast)
b) On merge to the
development
branch fuzzing tests should run with a great number of runs to test many cases (keep in mind that github action runnners can run for 6 hours)The text was updated successfully, but these errors were encountered: