Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MBT for ICS03 #701

Merged
merged 124 commits into from
Mar 2, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
124 commits
Select commit Hold shift + click to select a range
413d3f4
Add ICS02 model
vitorenesduarte Feb 2, 2021
065631e
Add MBT test driver
vitorenesduarte Feb 2, 2021
b98c7d3
Add ICS02TestExecutor
vitorenesduarte Feb 2, 2021
6685cc1
Add another apalache counterexample
vitorenesduarte Feb 2, 2021
f863486
Fix ICS02.tla: client counter starts at 0
vitorenesduarte Feb 2, 2021
cd0d0a6
Check for errors in MockContext.deliver
vitorenesduarte Feb 2, 2021
02c020b
Handle errors in MBT tests
vitorenesduarte Feb 2, 2021
a5d06c5
Remove SyntheticTendermint mock context
vitorenesduarte Feb 2, 2021
356c6d2
More idiomatic check_next_state
vitorenesduarte Feb 2, 2021
6ab8611
Buffered file reads
vitorenesduarte Feb 3, 2021
490d77b
Make extract_handler_error_kind generic over the IBC handler
vitorenesduarte Feb 3, 2021
92b843c
Support multiple chains in MBT
vitorenesduarte Feb 3, 2021
d57cf41
Make eyre a dev-dependency
vitorenesduarte Feb 3, 2021
f4dbe00
s/ICS0/IBC on TLA files
vitorenesduarte Feb 3, 2021
20867a5
Initial support for conn open init
vitorenesduarte Feb 3, 2021
e5e2f7b
Start connection and channel identifiers at 0
vitorenesduarte Feb 3, 2021
b9ca90a
Add conn open init to TLA spec
vitorenesduarte Feb 3, 2021
f3cd434
Represent clients with a record in TLA spec
vitorenesduarte Feb 3, 2021
4167d29
Finish connection open init
vitorenesduarte Feb 3, 2021
7755fec
s/state/step
vitorenesduarte Feb 3, 2021
70e70f6
Minimize diff
vitorenesduarte Feb 4, 2021
44747d3
Merge branch 'master' into vitor/ics02_mbt
vitorenesduarte Feb 4, 2021
41d34d3
Modularize TLA spec
vitorenesduarte Feb 4, 2021
1f9881c
TLA format convention
vitorenesduarte Feb 4, 2021
e2605bb
s/Null/None
vitorenesduarte Feb 4, 2021
5b98a7e
Sketch conn open try; Model chain's height
vitorenesduarte Feb 4, 2021
244efa6
Bound model space
vitorenesduarte Feb 4, 2021
7d9202a
Only update chain height upon success
vitorenesduarte Feb 4, 2021
28879f0
Check that chain heights match the ones in the model
vitorenesduarte Feb 4, 2021
a870537
Sketch conn open try
vitorenesduarte Feb 4, 2021
a533def
Sketch conn open try
vitorenesduarte Feb 4, 2021
1cdb02a
Model missing connections and connection mismatches in conn open try
vitorenesduarte Feb 4, 2021
0186a1c
Trigger bug in conn open try
vitorenesduarte Feb 4, 2021
f777144
Merge branch 'master' into vitor/ics02_mbt
vitorenesduarte Feb 4, 2021
6b6cd43
Go back to previous way of generating connection and channel ids
vitorenesduarte Feb 4, 2021
6186bec
Disable failing MBT test
vitorenesduarte Feb 4, 2021
a7b50d7
Fix panic in conn open try when no connection id is provided
vitorenesduarte Feb 5, 2021
5b66aac
ICS02TestExecutor -> IBCTestExecutor
vitorenesduarte Feb 5, 2021
095433e
Merge branch 'vitor/conn_open_try' into vitor/ics02_mbt
vitorenesduarte Feb 5, 2021
f2194cf
Failing MBT test now passes with #615
vitorenesduarte Feb 5, 2021
7bdd3b6
Add notes on MBT
vitorenesduarte Feb 8, 2021
9ee48bd
Remove ICS02
vitorenesduarte Feb 8, 2021
035651e
Add README
vitorenesduarte Feb 8, 2021
b6854ce
Improve README
vitorenesduarte Feb 8, 2021
cd484a6
Remove MBT intro
vitorenesduarte Feb 8, 2021
d69d15e
new lines
vitorenesduarte Feb 8, 2021
b4aea95
Merge branch 'master' into vitor/ics02_mbt
vitorenesduarte Feb 9, 2021
5e9dd63
Make MBT README more easily discoverable
vitorenesduarte Feb 9, 2021
fb20ce0
IBCTestExecutor: Map from ChainId (instead of String) to MockContext
vitorenesduarte Feb 9, 2021
75db6bb
s/epoch/revision
vitorenesduarte Feb 9, 2021
a102863
Move from eyre to thiserror
vitorenesduarte Feb 9, 2021
2e5d6de
Improve README
vitorenesduarte Feb 9, 2021
dfb683d
Improve README
vitorenesduarte Feb 9, 2021
7e24841
Improve arguments order in modelator::test
vitorenesduarte Feb 9, 2021
5d6bf08
Improve chain identifiers
vitorenesduarte Feb 10, 2021
60e4aba
Improve README
vitorenesduarte Feb 10, 2021
c700ba5
Start ICS03
vitorenesduarte Feb 10, 2021
c1206ae
Merge branch 'master' into vitor/mbt_ics03
vitorenesduarte Feb 11, 2021
7233987
Store all client heights and improve names in model output actions to…
vitorenesduarte Feb 11, 2021
ca8b839
Parse Action as an internally tagged enum
vitorenesduarte Feb 11, 2021
01b557b
merge master
vitorenesduarte Feb 12, 2021
b37982a
Separate action apply from action outcome checking
vitorenesduarte Feb 12, 2021
acbe0d8
improve conn open try enabling condition
vitorenesduarte Feb 12, 2021
28ee9a9
Fix IBCTests.tla
vitorenesduarte Feb 12, 2021
129ff9b
Add connectionProofs to each chain in TLA model
vitorenesduarte Feb 17, 2021
1d6d5e9
Generate actions in the handlers
vitorenesduarte Feb 18, 2021
00afd55
Save proofs in counterparty chain upon a successful action
vitorenesduarte Feb 18, 2021
8a2581d
Reduce state space
vitorenesduarte Feb 18, 2021
cf54ff4
Update tests
vitorenesduarte Feb 18, 2021
5e04972
Fix ICS02UpdateClient
vitorenesduarte Feb 18, 2021
a8023f2
Allow a conn open try to succeed only after a conn open init
vitorenesduarte Feb 19, 2021
e9af4de
Improve modelator's error
vitorenesduarte Feb 19, 2021
3575491
Generate tests
vitorenesduarte Feb 19, 2021
5170cc1
Add MockContext::host_oldest_height
vitorenesduarte Feb 19, 2021
3b83a64
Update CHANGELOG
vitorenesduarte Feb 19, 2021
fc6a238
Update CHANGELOG
vitorenesduarte Feb 19, 2021
5663b14
Merge branch 'vitor/overflow' into vitor/mbt_ics03
vitorenesduarte Feb 19, 2021
4da1d34
Fix conn open ack tests
vitorenesduarte Feb 19, 2021
56382c4
Update CHANGELOG.md
vitorenesduarte Feb 19, 2021
c851c33
Remove MockContext::host_chain_history_size
vitorenesduarte Feb 19, 2021
9b6d29a
Merge branch 'vitor/overflow' of https://github.com/informalsystems/i…
vitorenesduarte Feb 19, 2021
f4e8c27
Fix clippy
vitorenesduarte Feb 19, 2021
7fa9932
Fix clippy
vitorenesduarte Feb 19, 2021
d5aa60a
Merge branch 'vitor/overflow' into vitor/mbt_ics03
vitorenesduarte Feb 19, 2021
b4f31b2
Fix ICS03_ConnectionOpenTry
vitorenesduarte Feb 19, 2021
4bd0ea9
Finish connection open try
vitorenesduarte Feb 19, 2021
87bd1d7
Start conn open ack
vitorenesduarte Feb 19, 2021
b1115a9
merge master
vitorenesduarte Feb 19, 2021
daccb9b
merge master
vitorenesduarte Feb 19, 2021
f751e0e
handle ICS03ConnectionOpenAckOK
vitorenesduarte Feb 19, 2021
04b60bb
Merge branch 'master' into vitor/overflow
vitorenesduarte Feb 19, 2021
a2f40c8
merge master
vitorenesduarte Feb 19, 2021
1c6bd3a
Add conn open ack
vitorenesduarte Feb 20, 2021
ff4fbde
potential bug
vitorenesduarte Feb 22, 2021
ebb8569
Finish conn open ack
vitorenesduarte Feb 22, 2021
5f10217
Add conn open confirm
vitorenesduarte Feb 22, 2021
5a051af
Fix conn open ack
vitorenesduarte Feb 22, 2021
122210b
add conn open ack and conn open confirm failing tests
vitorenesduarte Feb 22, 2021
ac70ba3
updated height on successful conn open confirm
vitorenesduarte Feb 22, 2021
1d63ca9
Allow a conn open ack to succeed
vitorenesduarte Feb 22, 2021
b889b7b
Remove invalid test
vitorenesduarte Feb 22, 2021
abc29d8
merge vitor/open-ack
vitorenesduarte Feb 22, 2021
52a3390
enable all tests
vitorenesduarte Feb 22, 2021
0ef4b5a
merge master
vitorenesduarte Feb 22, 2021
a173dcd
update CHANGELOG
vitorenesduarte Feb 22, 2021
d25cc7b
fix CHANGELOG
vitorenesduarte Feb 22, 2021
96c4c6e
Merge branch 'vitor/open-ack' into vitor/mbt_ics03
vitorenesduarte Feb 22, 2021
067fc0c
fix CHANGELOG
vitorenesduarte Feb 22, 2021
809effb
Minimize diff
vitorenesduarte Feb 22, 2021
a3e95d3
Minimize diff
vitorenesduarte Feb 22, 2021
bf7c81d
Update README
vitorenesduarte Feb 22, 2021
731dcc9
Fix clippy
vitorenesduarte Feb 22, 2021
7269ee2
Merge branch 'vitor/open-ack' into vitor/mbt_ics03
vitorenesduarte Feb 22, 2021
ce23256
update README
vitorenesduarte Feb 22, 2021
6c7ee61
update README
vitorenesduarte Feb 22, 2021
53e2a08
Make MsgConnectionOpenAck.counterparty_connection_id not an Option
vitorenesduarte Feb 22, 2021
93a133b
Merge branch 'vitor/open-ack' into vitor/mbt_ics03
vitorenesduarte Feb 22, 2021
dda8c6f
Make MsgConnectionOpenAck.counterparty_connection_id not an Option
vitorenesduarte Feb 22, 2021
40adce4
Merge branch 'vitor/open-ack' into vitor/mbt_ics03
vitorenesduarte Feb 22, 2021
a8dbfe1
Fix test executor
vitorenesduarte Feb 22, 2021
0c5e9e4
Update README.md
vitorenesduarte Feb 22, 2021
eabf89d
Update README.md
vitorenesduarte Feb 22, 2021
4b0910a
merge master
vitorenesduarte Feb 23, 2021
6b66f7e
Uninit -> Uninitialized
vitorenesduarte Feb 23, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 16 additions & 18 deletions modules/tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,11 @@
This directory contains the model-based tests for the IBC modules. They are "model-based" because they are generated from a `TLA+` model of the IBC modules (see [IBC.tla](support/model_based/IBC.tla)).

To instantiate the model, we define in [IBC.cfg](support/model_based/IBC.cfg) the following model constants:
- `ChainIds = {"chain-A", "chain-B"}`, indicating that two chains, `chain-A` and `chain-B`, will be created
- `MaxClientsPerChain = 1`, indicating that at most 1 client per chain will be created
- `MaxClientHeight = 2`, indicating that clients will reach at most height 2

- `ChainIds = {"chain-A", "chain-B"}`, indicating that two chains, `chain-A` and `chain-B`, will be created
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit, but I'd suggest the same improvement as before 5d6bf08

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change in 5d6bf08 was reverted because the model doesn't know about revisions. Also, when generating the actual ChainId on the Rust side, we would get "chain-A-0-0":
https://github.com/informalsystems/ibc-rs/blob/3e513dbfd96fca6de0b2507bab5baff936b9864d/modules/tests/executor/mod.rs#L96-L98

- `MaxChainHeight = 4`, indicating that each chain will reach at most height 4
- `MaxClientsPerChain = 1`, indicating that at most 1 client per chain will be created
- `MaxConnectionsPerChain = 1`, indicating that at most 1 connection per chain will be created

The [IBC.cfg](support/model_based/IBC.cfg) file also defines two simple invariants:
```tla
Expand All @@ -16,13 +18,13 @@ INVARIANTS
ModelNeverErrors
```

Then, we can ask [`Apalache`](https://apalache.informal.systems), a model checker for `TLA+`, to check that these invariants hold:
Then, we can ask [`TLC`](https://github.com/tlaplus/tlaplus), a model checker for `TLA+`, to check that these invariants hold:

```bash
apalache-mc check --inv=ICS02UpdateOKTestNeg IBC.tla
wget https://github.com/tlaplus/tlaplus/releases/download/v1.8.0/tla2tools.jar
java -cp tla2tools.jar tlc2.TLC IBC.tla -tool -modelcheck -config IBC.cfg -workers auto
```

The above command automatically reads what we have defined in [IBC.cfg](support/model_based/IBC.cfg).

### The tests

Tests are `TLA+` assertions that describe the desired shape of the test (see [IBCTests.tla](support/model_based/IBCTests.tla)). One of the assertions in [IBCTests.tla](support/model_based/IBCTests.tla) is the following:
Expand All @@ -39,25 +41,21 @@ To generate a test from the `ICS02UpdateOKTest` assertion, we first define an in
ICS02UpdateOKTestNeg == ~ICS02UpdateOKTest
```

Then, we ask `Apalache`, to prove it:
Then, we ask `TLC`, to prove it. Because the invariant is wrong, `TLC` will find a counterexample showing that it is indeed possible that a client is sucessfully updated to a new height. This counterexample is our test.

The [`gen_tests.py`](support/model_based/gen_tests.py) script can be used to generate the tests. This script assumes the existence of [`tlc-json`](https://github.com/vitorenesduarte/tlc-json), which can be installed with the following commands:

```bash
apalache-mc check --inv=ICS02UpdateOKTestNeg IBCTests.tla
git clone https://github.com/vitorenesduarte/tlc-json
cd tlc-json/
cargo install --path .
```

(Again, the above command automatically reads what we have defined in [IBCTests.cfg](support/model_based/IBCTests.cfg).)

Because the invariant is wrong, `Apalache` will find a counterexample showing that it is indeed possible that a client is sucessfully updated to a new height. This counterexample is our test.

### Current limitations

The counterexamples currently produced by `Apalache` are not easy to parse and have traditionally required tools like [`jsonatr`](https://github.com/informalsystems/jsonatr). Fortunately, [that will change soon](https://github.com/informalsystems/apalache/issues/530), and `Apalache` will be able to produce counterexamples like those in [support/model_based/tests/](support/model_based/tests/).
These are currently generated manually, but can be easily mapped to Rust (see [step.rs](step.rs)).

### Running the model-based tests

The model-based tests can be run with the following command:

```bash
cd modules/
cargo test --features mocks -- mbt
```
Loading