Test Plan For Vat Upgrades

[anilhelvaci]

What is the Problem Being Solved?

Upgrade 19 is planned to include to upgrading the vats below;

• Vat Upgrade: v6-agoricNames #10408
• Vat upgrade: 14 mintHolders #10410
• Contract Upgrade: V28 ProvisionPool and V14 Bank #10395
• Contract Upgrade: v23-feeDistributor #10393
• Contract Upgrade: v7-board #10394
• Vat Upgrade: v8-PriceAuthority #10401
• Contract Upgrade: v36-Reserve #10399

On-going

• Contract Upgrade: 8 PSM contracts #10403

We need a plan for testing all these vats above in Emerynet and any other test nets that are required. These test plans should take into account the differences of testnets compared to a local a3p node;

• core eval voting cannot be manipulated as freely
  • for testnet like xnet it might be possible to take over the voting power and run core evals that set up some required state for testing however for a more formal mainnet fork like Emerynet I suspect even that might not be possible
• funding accounts are more limited compared to doing a bankSend in a3p as to the nature of the faucet available

Description of the Design

Filling the issued listed below with a test scenario and its form of execution (manually, with a script or a mix of both).

Tasks

Preview Give feedback

1. Test Plan: v6-agoricNames upgrade #10647
   enhancement +1
   
2. Test Plan: mintHolders upgrade #10648
   enhancement +1
   
3. Test Plan: v28-provisionPool and v14-bank upgrade #10649
   enhancement +1
   
4. Test Plan: v23-feeDistributor upgrade #10650
   enhancement +1
   
5. Test Plan: PSM contracts upgrade #10651
   enhancement +1
   
6. Test Plan: v7-board upgrade #10652
   enhancement +1
   
7. Test Plan: v8-priceAuthority upgrade #10653
   enhancement +1
   
8. Test Plan: v36-reserve upgrade #10688
   +0
   

Security Considerations

Having a comprehensive test coverage is crucial for the chain's health. So we should pay the highest attention that we can when building the test scenarios.

Scaling Considerations

Since the vats being upgraded form a part of the agoric chain's foundation in terms of functionality, any error here would block further improvement.

Test Plan

Build test plans for each individual vat. If there are occasions where multiple vats are interconnected in terms of testing their functionality, we might consider coupling those together in one test scenario.

For convenience, I believe it would be useful to have a pool of accounts to use as our clients when testing. Also for the operations that we plan to carry out using scripts, we probably need a place to put that code. Maybe a new and smalll repository? Not sure how this could go down. Could use some help @LuqiPan

[anilhelvaci]

Common Operations

We see that there are some common operations we will be using to test some of the vats above. For example; opening a vault is common for the vats below;

• vat-agoricNames
• vat-provisionPool (auto provisioning)
• vat-feeDistributor

Note that there might be other vats added to the list above

Should we merge the tests that are common across multiple vats?

• Pros
  • less steps to follow
• Cons
  • in case of a bug it might get tricky to debug as what to test isn't isolated

cc @Chris-Hibbert

[anilhelvaci]

How much weight should be given to tests that require a core-eval to setup?

Some tests require core evals to setup its environment. For example, to write to agoricNames, a core eval required since we don't have agoricNamesAdmin anywhere other than bootstrap space.

To execute a core eval in test nets MIGHT present a problem, depending on the test we are running our tests against. The conditions that enable running core-evals for testing;

• We should control the voting power of BLDer DAO
• The voting period should be reasonable, seconds or minutes instead of days

For test networks above conditions hold, it's possible to include the tests that require a core-eval.

Let's say above conditions hold for Xnet and does not hold for Emerynet (I suspect this is truly the case). What should we do?

Note

The tests that require core eval are already included in a3p.

cc @LuqiPan @Chris-Hibbert

[Jorge-Lopes]

Common test happy path (WIP)

High level test plan

Test steps:

# [ agd ] 
> Add new wallet 
> Provision wallet
> Verify wallet on vstorage

# [ core-eval ] 
> Add new asset
> Create mintHolder 
> Write to agoricNames
> Start new PSM
 
# [ agd ] 
> Verify agoricNames
> Provision provisionPoolAddress

# [ core-eval ] 
> Fetch mintHolder contractKit
> Mint payment
> Fetch provisionPoolAddress deposit facet
> Fund address

# [ agd ]
> Verify bank balance
> Open new vault
> Verify reserve metrics

cc: @anilhelvaci

[Chris-Hibbert]

Should we merge the tests that are common across multiple vats?

Another consideration that pushes us more toward duplication is that the tests sometimes get run separately. In that case, it's very useful to have the test duplicated. The usual approach to this is to just expect duplication, but when we find an individual test that takes a long time, we'll look for ways to combine copies of it.

[Chris-Hibbert]

How much weight should be given to tests that require a core-eval to setup?

This goes in the category of manual testing. I don't think we need to repeat all the tests in every environment. If there's a reason to expect issues, it might be worth checking, but if we can write (and follow) a test plan that provides evidence that the upgrades we wanted actually ran, we don't need to re-test all the functionality.

Opinions, @otoole-brendan, @LuqiPan @turadg?