Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI flake: "provided port is already allocated" occassionally happens during e2e cross cluster tests #2357

Open
Tracked by #2512
AustinAbro321 opened this issue Mar 5, 2024 · 3 comments

Comments

@AustinAbro321
Copy link
Contributor

We have seen this flake a few times, it is unclear why it is happening.
In all of the cases the error is provided port is already allocated
In the following two cases the error is on the below line.

_, initStdErr, err := e2e.Zarf("init", "--components="+initComponents, "--nodeport", "31337", "-l", "trace", "--confirm")
require.NoError(t, err)

We also see a similar error here on here on test 99:

stdOut, stdErr, err = e2e.Zarf("init", "--components=k3s", "--confirm")
require.NoError(t, err, stdOut, stdErr)
@AustinAbro321 AustinAbro321 changed the title "provided port is already allocated" test flake can occassionally happen during e2e cross cluster tests CI flake: "provided port is already allocated" occassionally happens during e2e cross cluster tests Mar 5, 2024
@AustinAbro321
Copy link
Contributor Author

We believe that during the first two occurrences of the the problem was the registry port 31337 assigned in the test conflicted with the randomly generated trafeik ports created by k3d. This will be solved by disabling trafeik during e2e tests in #2282.

The third issue is an actual bug in zarf. It is caused by the fact that the injector (seed registry) selects a random port and this port could conflict with the registry port. Looking at the logs here confirms this as the seed registry matches the registry in the logs https://github.com/defenseunicorns/zarf/actions/runs/8012041029/job/21886586280.

@Noxsios Noxsios moved this to Backlog in Zarf (old) Mar 19, 2024
@lucasrod16
Copy link
Contributor

@AustinAbro321
Copy link
Contributor Author

Yeah it's only like a 1 in 3000 chance, but it's still worth fixing for the init case. Especially since this can also happen to real uses if they have the proper luck or lack thereof

@salaxander salaxander added this to Zarf Jul 22, 2024
@github-project-automation github-project-automation bot moved this to Backlog in Zarf Jul 22, 2024
@salaxander salaxander removed the status in Zarf Jul 22, 2024
@salaxander salaxander moved this to Triage in Zarf Sep 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Triage
Development

No branches or pull requests

2 participants