Flaky E2E Tests

[thewhaleking]

Test frequently fails for completely unrelated PRs.

test_commit_and_reveal_weights_cr3 fails often
test_liqudity fails about half the time

CR3 failure:

          # Fetch current commits pending on the chain
          commits_on_chain = subtensor.get_current_weight_commit_info_v2(netuid=netuid)
          address, commit_block, commit, reveal_round = commits_on_chain[0]
      
          # Assert correct values are committed on the chain
          assert expected_reveal_round == reveal_round
          assert address == alice_wallet.hotkey.ss58_address
  >       assert commit_block == expected_commit_block + 1
  E       assert 38 == (38 + 1)

Liquidity failure:

          # Add liquidity
          success, message = subtensor.extrinsics.add_liquidity(
              wallet=alice_wallet,
              netuid=alice_subnet_netuid,
              liquidity=Balance.from_tao(1),
              price_low=Balance.from_tao(1.7),
              price_high=Balance.from_tao(1.8),
              wait_for_inclusion=True,
              wait_for_finalization=True,
          )
  >       assert success, message
  E       AssertionError: Subtensor returned `InsufficientBalance(Module)` error. This means: `The caller does not have enough balance for the operation. | Please consult [https://docs.bittensor.com/errors/subtensor#insufficientbalance`.](https://docs.bittensor.com/errors/subtensor#insufficientbalance%60.)
  E       assert False

[thewhaleking]

This may only be on the devnet-ready image. Testing further.

It's both

[thewhaleking]

Note that while these fail frequently in the GH Actions, I can't actually get test_liquidity to ever fail running locally.

crv3 fails ~⅓-¼ of the time running locally.

[thewhaleking]

Why do we do this?

assert commit_block == expected_commit_block + 1

Again, ~⅓ of the time commit_block == expected_commit_block, with the other ⅔ being commit_block == expected_commit_block + 1

[basfroman]

Why do we do this?

assert commit_block == expected_commit_block + 1
Again, ~⅓ of the time commit_block == expected_commit_block, with the other ⅔ being commit_block == expected_commit_block + 1

I've improved this logic. The thing is that expected_commit_block is obtained when we set weights. commit_block is returned by the chain based on the block when the extrinsic is executed (on the chain). Due to the speed of fast runtime, the block can vary +1 or the same block from expected_commit_block. Now it works. At least, I ran it locally 20 times :D and not a single test failed.