Invariant Testing #563
Comments
0x4007
added
DevOps
|
We would have to determine if the protocol has any hard and fast rules like Uniswap's x*y=k. I don't think it's useful for the entire protocol but if we can find any "rules" within the protocol that we would expect to persist no matter the conditions then invariant testing would be useful to stress test those. |
|
Perhaps @zgorizzo69 might have some ideas. I haven't personally done the deep dive on what exactly invariant testing is but after I do I would have an opinion on it. |
|
/start |
Tips:
|
|
@pavlovcik is this still open |
|
I think the research is useful but it's @rndquu's decision if this bounty is invalidated. |
I would close the following tasks (including the current issue):
All of them will be part of a security epic which we will break down into small tasks @pavlovcik ok? |
|
Just to clarify, you want to create a meta-issue with these as new subtasks? What's the benefit of remaking all of them vs just manually compiling them in a single new issue? |
yes
I don't really understand the question. We can't compile all of them in a single issue because each of those tasks is pretty time consuming. |
|
Do you have any updates @malik672? If you would like to release the bounty back to the DevPool, please comment |
I meant creating a new blank issue, and then linking all of the existing sub issues in a checklist. The blank issue is now your "meta-issue" The alternative is that you mark all of the issues as unplanned, then create a blank issue with a checklist, and then recreate all the sub tasks. Both will produce the same outcome but one should be much faster to do, if it works like that. |
I meant the same flow |
Can you handle this @rndquu ? |
I didn't forget about this task We first need to remove mocks from the contracts because it doesn't make sense in fuzzing Dollar contracts which use mocks under the hood I'll break down the "remove mocks" epic into subtasks soon |
|
/start |
|
Hey everyone, I've also started the UI testing issue #257 |
|
You can be assigned to both. |
|
Hi @0x4007, @rndquu, and team, I wanted to follow up regarding my pull request that addresses this issue. I wanted to check if there are any updates or further changes needed from my end to move this forward. Any feedback or guidance would be greatly appreciated |
Hey, sorry for a long response, I'll check it today |
|
/start |
! alexandr-masl you were previously unassigned from this task. You cannot be reassigned. |
|
I have been unassigned from the issue for some reason. Could you please provide any clarification ? |
As far as I understand the |
|
@alexandr-masl the deadline is at Sat, Sep 14, 8:43 PM UTC |
I think it doesn't read draft pulls right now. This will be fixed soon. Also strange it didn't warn at 3.5 days. @gentlementlegen |
|
@alexandr-masl, this task has been idle for a while. Please provide an update. |
|
Hey, please let me know if any further changes are expected from my side to move this forward |
|
@alexandr-masl, this task has been idle for a while. Please provide an update. |
|
| View | Contribution | Count | Reward |
|---|---|---|---|
| Issue | Task | 1 | 300 |
| Issue | Comment | 4 | 0.78 |
| Review | Comment | 14 | 0 |
Conversation Incentives
| Comment | Formatting | Relevance | Reward |
|---|---|---|---|
Hey everyone, I've also started the UI testing issue [#257](http… | 2.4content:
p:
symbols:
\b\w+\b:
count: 40
multiplier: 0.1
score: 1
a:
symbols:
\b\w+\b:
count: 1
multiplier: 0.1
score: 1
multiplier: 1
| 0.1 | 0.24 |
Hi @0x4007, @rndquu, and team, I wanted to follow up regarding … | 2.7content:
p:
symbols:
\b\w+\b:
count: 43
multiplier: 0.1
score: 1
a:
symbols:
\b\w+\b:
count: 3
multiplier: 0.1
score: 1
multiplier: 1
| 0.2 | 0.54 |
I have been unassigned from the issue for some reason. Could you… | 0content:
p:
symbols:
\b\w+\b:
count: 16
multiplier: 0.1
score: 1
multiplier: 1
| - | - |
Hey, please let me know if any further changes are expected from… | 0content:
p:
symbols:
\b\w+\b:
count: 18
multiplier: 0.1
score: 1
multiplier: 1
| - | - |
Resolves #563 | 0content:
p:
symbols:
\b\w+\b:
count: 2
multiplier: 0
score: 1
multiplier: 0
| 0.2 | - |
Thank you for the feedback! I'm currently experimenting with the… | 0content:
p:
symbols:
\b\w+\b:
count: 21
multiplier: 0.2
score: 1
a:
symbols:
\b\w+\b:
count: 3
multiplier: 0.2
score: 1
multiplier: 0
| 0.7 | - |
The invocation of setCollateralRatio has been completely removed… | 0content:
p:
symbols:
\b\w+\b:
count: 11
multiplier: 0.2
score: 1
multiplier: 0
| 0.8 | - |
The setRedemptionDelay() method has been removed | 0content:
p:
symbols:
\b\w+\b:
count: 6
multiplier: 0.2
score: 1
multiplier: 0
| 0.8 | - |
The vm.prank(user) has been added | 0content:
p:
symbols:
\b\w+\b:
count: 7
multiplier: 0.2
score: 1
multiplier: 0
| 0.7 | - |
Yes, as mentioned earlier, the invocation of setCollateralRatio … | 0content:
p:
symbols:
\b\w+\b:
count: 28
multiplier: 0.2
score: 1
multiplier: 0
| 0.7 | - |
Yes, it wasn't the best idea. It was initially used during testi… | 0content:
p:
symbols:
\b\w+\b:
count: 29
multiplier: 0.2
score: 1
multiplier: 0
| 0.6 | - |
Ok, will upgrade it I'm considering combining both the invarian… | 0content:
p:
symbols:
\b\w+\b:
count: 101
multiplier: 0.2
score: 1
code:
symbols:
\b\w+\b:
count: 2
multiplier: 0.2
score: 1
a:
symbols:
\b\w+\b:
count: 4
multiplier: 0.2
score: 1
multiplier: 0
| 0.8 | - |
got it, will refactor | 0content:
p:
symbols:
\b\w+\b:
count: 4
multiplier: 0.2
score: 1
multiplier: 0
| 0.4 | - |
**Hey everyone,** I believe we're could start with this invaria… | 0content:
p:
symbols:
\b\w+\b:
count: 128
multiplier: 0.2
score: 1
strong:
symbols:
\b\w+\b:
count: 2
multiplier: 0.2
score: 0
ol:
symbols:
\b\w+\b:
count: 1
multiplier: 0.2
score: 0
li:
symbols:
\b\w+\b:
count: 2
multiplier: 0.2
score: 1
multiplier: 0
| 0.9 | - |
Thanks for the feedback and kind words, will come up with an upd… | 0content:
p:
symbols:
\b\w+\b:
count: 14
multiplier: 0.2
score: 1
multiplier: 0
| 0.3 | - |
Thanks for the feedback. I'm actively working on it | 0content:
p:
symbols:
\b\w+\b:
count: 10
multiplier: 0.2
score: 1
multiplier: 0
| 0.3 | - |
Please review my basic invariant tests. I have minimized the use… | 0content:
p:
symbols:
\b\w+\b:
count: 69
multiplier: 0.2
score: 1
code:
symbols:
\b\w+\b:
count: 2
multiplier: 0.2
score: 1
ul:
symbols:
\b\w+\b:
count: 1
multiplier: 0.2
score: 1
li:
symbols:
\b\w+\b:
count: 2
multiplier: 0.2
score: 1
multiplier: 0
| 0.9 | - |
I'm not highly experienced with the `foundry.toml` profi… | 0content:
p:
symbols:
\b\w+\b:
count: 51
multiplier: 0.2
score: 1
code:
symbols:
\b\w+\b:
count: 57
multiplier: 0.2
score: 1
a:
symbols:
\b\w+\b:
count: 2
multiplier: 0.2
score: 1
pre:
symbols:
\b\w+\b:
count: 1
multiplier: 0.2
score: 0
multiplier: 0
| 0.7 | - |
[ 0.656 WXDAI ]
@hashedMae
Contributions Overview
| View | Contribution | Count | Reward |
|---|---|---|---|
| Issue | Comment | 1 | 0.656 |
Conversation Incentives
| Comment | Formatting | Relevance | Reward |
|---|---|---|---|
We would have to determine if the protocol has any hard and fast… | 0.82content:
p:
symbols:
\b\w+\b:
count: 61
multiplier: 0.1
score: 1
multiplier: 0.25
| 0.8 | 0.656 |
[ 68.782 WXDAI ]
@0x4007
Contributions Overview
| View | Contribution | Count | Reward |
|---|---|---|---|
| Issue | Specification | 1 | 63.53 |
| Issue | Comment | 7 | 3.907 |
| Review | Comment | 2 | 1.345 |
Conversation Incentives
| Comment | Formatting | Relevance | Reward |
|---|---|---|---|
We should implement invariant testing for [LibUbiquityPool](http… | 63.53content:
p:
symbols:
\b\w+\b:
count: 194
multiplier: 0.1
score: 1
a:
symbols:
\b\w+\b:
count: 3
multiplier: 0.1
score: 5
ul:
symbols:
\b\w+\b:
count: 2
multiplier: 0.1
score: 0
li:
symbols:
\b\w+\b:
count: 8
multiplier: 0.1
score: 1
ol:
symbols:
\b\w+\b:
count: 1
multiplier: 0.1
score: 0
code:
symbols:
\b\w+\b:
count: 36
multiplier: 0.1
score: 5
pre:
symbols:
\b\w+\b:
count: 1
multiplier: 0.1
score: 0
multiplier: 3
| 1 | 63.53 |
Perhaps @zgorizzo69 might have some ideas. I haven't personally … | 3.7content:
p:
symbols:
\b\w+\b:
count: 31
multiplier: 0.2
score: 1
multiplier: 1
| 0.3 | 1.11 |
I think the research is useful but it's @rndquu's decision if th… | 2.22content:
p:
symbols:
\b\w+\b:
count: 17
multiplier: 0.2
score: 1
multiplier: 1
| 0.1 | 0.222 |
Just to clarify, you want to create a meta-issue with these as n… | 4.01content:
p:
symbols:
\b\w+\b:
count: 34
multiplier: 0.2
score: 1
multiplier: 1
| 0.2 | 0.802 |
I meant creating a new blank issue, and then linking all of the … | 8.47content:
h2:
symbols:
\b\w+\b:
count: 27
multiplier: 0.2
score: 1
p:
symbols:
\b\w+\b:
count: 46
multiplier: 0.2
score: 1
multiplier: 1
| 0.2 | 1.694 |
Can you handle this @rndquu ? | 0.79content:
p:
symbols:
\b\w+\b:
count: 5
multiplier: 0.2
score: 1
multiplier: 1
| 0.1 | 0.079 |
You can be assigned to both. | 0content:
p:
symbols:
\b\w+\b:
count: 6
multiplier: 0.2
score: 1
multiplier: 1
| - | - |
I think it doesn't read draft pulls right now. This will be fixe… | 0content:
p:
symbols:
\b\w+\b:
count: 26
multiplier: 0.2
score: 1
multiplier: 1
| - | - |
Welcome back phoenix | 0.25content:
p:
symbols:
\b\w+\b:
count: 3
multiplier: 0.1
score: 1
multiplier: 1
| 0.1 | 0.025 |
I understand that this is a draft pull request, so you should ma… | 1.65content:
p:
symbols:
\b\w+\b:
count: 27
multiplier: 0.1
score: 1
multiplier: 1
| 0.8 | 1.32 |
[ 0.01 WXDAI ]
@malik672
Contributions Overview
| View | Contribution | Count | Reward |
|---|---|---|---|
| Issue | Comment | 1 | 0.01 |
Conversation Incentives
| Comment | Formatting | Relevance | Reward |
|---|---|---|---|
@pavlovcik is this still open | 0.1content:
p:
symbols:
\b\w+\b:
count: 5
multiplier: 0.1
score: 1
multiplier: 0.25
| 0.1 | 0.01 |
[ 20.715 WXDAI ]
@rndquu
Contributions Overview
| View | Contribution | Count | Reward |
|---|---|---|---|
| Issue | Comment | 7 | 3.811 |
| Review | Comment | 14 | 16.904 |
Conversation Incentives
| Comment | Formatting | Relevance | Reward |
|---|---|---|---|
I would close the following tasks (including the current issue): … | 3.85content:
p:
symbols:
\b\w+\b:
count: 60
multiplier: 0.1
score: 1
ul:
symbols:
\b\w+\b:
count: 1
multiplier: 0.1
score: 1
li:
symbols:
\b\w+\b:
count: 4
multiplier: 0.1
score: 1
a:
symbols:
\b\w+\b:
count: 2
multiplier: 0.1
score: 1
multiplier: 1
| 0.2 | 0.77 |
yes I don't really understand the question. We can't compile al… | 0content:
p:
symbols:
\b\w+\b:
count: 28
multiplier: 0.1
score: 1
multiplier: 1
| - | - |
I meant the same flow | 0content:
p:
symbols:
\b\w+\b:
count: 5
multiplier: 0.1
score: 1
multiplier: 1
| - | - |
I didn't forget about this task We first need to remove mocks f… | 2.45content:
p:
symbols:
\b\w+\b:
count: 43
multiplier: 0.1
score: 1
multiplier: 1
| 0.4 | 0.98 |
Invariant testing is a subpart of fuzzing tests which we already… | 2.48content:
p:
symbols:
\b\w+\b:
count: 38
multiplier: 0.1
score: 1
a:
symbols:
\b\w+\b:
count: 1
multiplier: 0.1
score: 1
code:
symbols:
\b\w+\b:
count: 2
multiplier: 0.1
score: 1
multiplier: 1
| 0.8 | 1.984 |
Hey, sorry for a long response, I'll check it today | 0.77content:
p:
symbols:
\b\w+\b:
count: 11
multiplier: 0.1
score: 1
multiplier: 1
| 0.1 | 0.077 |
As far as I understand the `ubiquity-os` bot unassigned … | 0content:
p:
symbols:
\b\w+\b:
count: 16
multiplier: 0.1
score: 1
code:
symbols:
\b\w+\b:
count: 2
multiplier: 0.1
score: 1
multiplier: 1
| - | - |
@alexandr-masl Overall this is a great job. 1. Pls fix minor is… | 3.64content:
p:
symbols:
\b\w+\b:
count: 39
multiplier: 0.1
score: 1
ol:
symbols:
\b\w+\b:
count: 1
multiplier: 0.1
score: 0
li:
symbols:
\b\w+\b:
count: 3
multiplier: 0.1
score: 1
a:
symbols:
\b\w+\b:
count: 3
multiplier: 0.1
score: 1
code:
symbols:
\b\w+\b:
count: 13
multiplier: 0.1
score: 1
multiplier: 1
| 0.8 | 2.912 |
Pls add this config to [foundry.toml](https://github.com/ubiquit… | 1.1content:
p:
symbols:
\b\w+\b:
count: 5
multiplier: 0.1
score: 1
a:
symbols:
\b\w+\b:
count: 2
multiplier: 0.1
score: 1
pre:
symbols:
\b\w+\b:
count: 1
multiplier: 0.1
score: 0
code:
symbols:
\b\w+\b:
count: 7
multiplier: 0.1
score: 1
multiplier: 1
| 0.7 | 0.77 |
Overall this is a good start. There're still too many reverts bu… | 1.65content:
p:
symbols:
\b\w+\b:
count: 27
multiplier: 0.1
score: 1
multiplier: 1
| 0.6 | 0.99 |
I know it's still a draft but it would be super cool to use diff… | 1.74content:
p:
symbols:
\b\w+\b:
count: 24
multiplier: 0.1
score: 1
a:
symbols:
\b\w+\b:
count: 3
multiplier: 0.1
score: 1
multiplier: 1
| 0.9 | 1.566 |
This line always reverts because `collateralRatio` [can'… | 1.15content:
p:
symbols:
\b\w+\b:
count: 9
multiplier: 0.1
score: 1
code:
symbols:
\b\w+\b:
count: 2
multiplier: 0.1
score: 1
a:
symbols:
\b\w+\b:
count: 4
multiplier: 0.1
score: 1
multiplier: 1
| 0.9 | 1.035 |
This handler method adds too much unnecessary reverts on collect… | 3.21content:
p:
symbols:
\b\w+\b:
count: 43
multiplier: 0.1
score: 1
ol:
symbols:
\b\w+\b:
count: 1
multiplier: 0.1
score: 0
li:
symbols:
\b\w+\b:
count: 2
multiplier: 0.1
score: 1
code:
symbols:
\b\w+\b:
count: 3
multiplier: 0.1
score: 1
a:
symbols:
\b\w+\b:
count: 4
multiplier: 0.1
score: 1
multiplier: 1
| 0.8 | 2.568 |
Shouldn't we use `vm.prank(user)` here? | 0.65content:
p:
symbols:
\b\w+\b:
count: 5
multiplier: 0.1
score: 1
code:
symbols:
\b\w+\b:
count: 3
multiplier: 0.1
score: 1
multiplier: 1
| 0.7 | 0.455 |
Also may revert in some cases because collateral ratio can't be … | 1.9content:
p:
symbols:
\b\w+\b:
count: 25
multiplier: 0.1
score: 1
code:
symbols:
\b\w+\b:
count: 2
multiplier: 0.1
score: 1
a:
symbols:
\b\w+\b:
count: 2
multiplier: 0.1
score: 1
multiplier: 1
| 0.7 | 1.33 |
Why do we `div(2)` here? What's the difference between &… | 1.65content:
p:
symbols:
\b\w+\b:
count: 14
multiplier: 0.1
score: 1
code:
symbols:
\b\w+\b:
count: 10
multiplier: 0.1
score: 1
multiplier: 1
| 0.6 | 0.99 |
Invariant and fuzz CI workflows are kept in separate files becau… | 3.66content:
p:
symbols:
\b\w+\b:
count: 69
multiplier: 0.1
score: 1
multiplier: 1
| 0.8 | 2.928 |
```suggestion * @notice Manipulates the stable… | 0.46content:
pre:
symbols:
\b\w+\b:
count: 1
multiplier: 0.1
score: 0
code:
symbols:
\b\w+\b:
count: 6
multiplier: 0.1
score: 1
multiplier: 1
| 0.6 | 0.276 |
```suggestion * @notice Manipulates the collat… | 0.39content:
pre:
symbols:
\b\w+\b:
count: 1
multiplier: 0.1
score: 0
code:
symbols:
\b\w+\b:
count: 5
multiplier: 0.1
score: 1
multiplier: 1
| 0.6 | 0.234 |
```suggestion * @return totalDollarSupplyInUsd… | 0.83content:
pre:
symbols:
\b\w+\b:
count: 1
multiplier: 0.1
score: 0
code:
symbols:
\b\w+\b:
count: 12
multiplier: 0.1
score: 1
multiplier: 1
| 0.6 | 0.498 |
That's strange, we'll see how it goes in the CI environment. | 0.88content:
p:
symbols:
\b\w+\b:
count: 13
multiplier: 0.1
score: 1
multiplier: 1
| 0.4 | 0.352 |
[ 0.777 WXDAI ]
@molecula451
Contributions Overview
| View | Contribution | Count | Reward |
|---|---|---|---|
| Issue | Comment | 1 | 0.777 |
Conversation Incentives
| Comment | Formatting | Relevance | Reward |
|---|---|---|---|
@rndquu this taks has "DevOps" label it's this expected to perfo… | 1.11content:
p:
symbols:
\b\w+\b:
count: 17
multiplier: 0.1
score: 1
multiplier: 1
| 0.7 | 0.777 |
[ 0.408 WXDAI ]
@gitcoindev
Contributions Overview
| View | Contribution | Count | Reward |
|---|---|---|---|
| Review | Comment | 2 | 0.408 |
Conversation Incentives
| Comment | Formatting | Relevance | Reward |
|---|---|---|---|
Hi, I am slowly trying to raise like a phoenix from the ashes. I… | 0.55content:
p:
symbols:
\b\w+\b:
count: 33
multiplier: 0.1
score: 1
code:
symbols:
\b\w+\b:
count: 3
multiplier: 0.1
score: 1
multiplier: 0.25
| 0.6 | 0.33 |
I did a review of the invariant tests skeleton, looks like a goo… | 0.39content:
p:
symbols:
\b\w+\b:
count: 25
multiplier: 0.1
score: 1
multiplier: 0.25
| 0.2 | 0.078 |
We should implement invariant testing for LibUbiquityPool.
So collaborator who's going to take this issue should carefully study the LibUbiquityPool and think of the invariants that should be tested.
Invariants could be:
Possible solutions for invariant testing (I would simply start with foundry since we're using it as a development framework + it also contains fuzz testing):
What should be done:
UbiquityPoolFacet.invariant.t.solso we could distinguish unit, fuzz and invariant tests)a) On PR invariant tests should run with a small number of runs (so that we could merge PRs fast)
b) On merge to the
developmentbranch invariant tests should run with a great number of runs to test many cases (keep in mind that github action runnners can run for 6 hours)Original description:
The text was updated successfully, but these errors were encountered: