Speed up TestAppServersHA #12128
Speed up TestAppServersHA #12128
Conversation
| results := make(chan result, len(configs)) | ||
| for _, conf := range configs { | ||
| go func(cfg *service.Config) { | ||
| dataDir, err := os.MkdirTemp("", "cluster-"+i.Secrets.SiteName) |
There was a problem hiding this comment.
Can this logic reuse StartApp above? Seems like most of logic here is the same.
There was a problem hiding this comment.
I initially tried to launch StartApp in a goroutine for each app, but it turns out TeleInstance is racy. StartApp, StartNode, StartDatabase, etc all update i.tempDirs and i.Nodes without using a mutex. I opted to add StartApps and handle updating the TeleInstance fields in a way that wasn't racy instead of adding a mutex to TeleInstance and finding all the possible races.
The downside is as you mentioned StartApp and StartApps are nearly identical. If you think it would be better to remove StartApps and go down the mutex route I'm happy to make the change.
Allow test cases to be run in parrallel and allow app servers to be spawned in parrallel to reduce test time from ~99s to ~25s.
… processes required for the test
rosstimothy
force-pushed
the
tross/speed_up_app_servers_ha
branch
from
54f670e to
198d994
Compare
|
I further simplified the tests in 198d994 to hopefully get CI to pass. This also shaved some more time off the test: while go test ./integration -run TestAppServersHA -count=1 -race; do :; done
ok github.com/gravitational/teleport/integration 19.709s
ok github.com/gravitational/teleport/integration 19.889s
ok github.com/gravitational/teleport/integration 20.801s
ok github.com/gravitational/teleport/integration 20.360s
ok github.com/gravitational/teleport/integration 20.451s
ok github.com/gravitational/teleport/integration 19.464s
ok github.com/gravitational/teleport/integration 20.956s
ok github.com/gravitational/teleport/integration 20.930s
ok github.com/gravitational/teleport/integration 19.739s |
* Speed up TestAppServersHA Allow test cases to be run in parrallel and allow app servers to be spawned in parrallel to reduce test time from ~99s to ~20s. (cherry picked from commit 71dea2d)
* Speed up TestAppServersHA Allow test cases to be run in parrallel and allow app servers to be spawned in parrallel to reduce test time from ~99s to ~20s. (cherry picked from commit 71dea2d)
💚 All backports created successfully
Questions ?Please refer to the Backport tool documentation |
Allow test cases to be run in parallel and allow app servers to
be spawned in parallel to reduce test time from ~99s to ~25s.
From master:
With changes in this PR: