Restructure E2E tests to reduce number of runners used

[chatton]

Summary

Currently, our E2E tests use one host per runner, each test spins up a new chain, we run a test, and tear down the chain.

In order to save on runners, we can instead try to run 1 test suite per runner. Implementation wise the following may work

1. Setup of test suite would create docker resources
2. Setup of test would create a new channel/connection
3. Perform test as usual with the addition of t.Parallel()

We would need to watch out for tests that have chain-wide effects like anything using gov proposals etc.

Ideally we can still run individual tests the way we do now locally.

---

For Admin Use

• Not duplicate issue
• Appropriate labels applied
• Appropriate contributors tagged/assigned

[vuong177]

@chatton could I pick up this issue ?

[chatton]

@vuong177 sure! It might be nice to try and PoC this on a single test suite, maybe this one?

Let me know if you need a hand with any of the workflow files. Also if you're able to get it working locally, I'd be happy to help out with the CI integration.

[anhductn2001]

Hey @chatton, I'm here to solve it with @vuong177.
And I have a question: "Can we use a fork of strangelove/interchaintest? Because currently, interchaintest will require clean-up all containers after each test. And to achieve what we want in this issue, we need to remove this cleanup function.

func DockerSetup(t DockerSetupTestingT) (*client.Client, string) {
	t.Helper()

	cli, err := client.NewClientWithOpts(client.FromEnv)
	if err != nil {
		panic(fmt.Errorf("failed to create docker client: %v", err))
	}

	// Clean up docker resources at end of test.
	t.Cleanup(dockerCleanup(t, cli))

	// Also eagerly clean up any leftover resources from a previous test run,
	// e.g. if the test was interrupted.
	dockerCleanup(t, cli)()

	name := fmt.Sprintf("interchaintest-%s", RandLowerCaseLetterString(8))
	network, err := cli.NetworkCreate(context.TODO(), name, types.NetworkCreate{
		CheckDuplicate: true,

		Labels: map[string]string{CleanupLabel: t.Name()},
	})
	if err != nil {
		panic(fmt.Errorf("failed to create docker network: %v", err))
	}

	return cli, network.ID
}