Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Flaky TestFifo #2557

Open
AlexanderYastrebov opened this issue Aug 31, 2023 · 14 comments · Fixed by #2654
Open

Flaky TestFifo #2557

AlexanderYastrebov opened this issue Aug 31, 2023 · 14 comments · Fixed by #2654
Labels
flakytest

Comments

@AlexanderYastrebov
Copy link
Member 

--- FAIL: TestFifo (3.26s)
    --- FAIL: TestFifo/fifo_with_reaching_max_concurrency_and_queue_timeouts (1.03s)
        fifo_test.go:260: aroute: * -> fifo(3, 5, "10ms") -> "http://127.0.0.1:46699"
        fifo_test.go:306: Success [0..1]: 0.10
        fifo_test.go:307: requests: 1990
        fifo_test.go:311: OK rate too low got<want: 198 < 199
This was referenced Aug 31, 2023
Merged
Open
Merged
@AlexanderYastrebov
Copy link
Member Author 

--- FAIL: TestFifo (3.27s)
    --- FAIL: TestFifo/fifo_with_reaching_max_concurrency_and_queue_timeouts (1.04s)
        fifo_test.go:260: aroute: * -> fifo(3, 5, "10ms") -> "http://127.0.0.1:43963"
        fifo_test.go:306: Success [0..1]: 0.09
        fifo_test.go:307: requests: 1993
        fifo_test.go:311: OK rate too low got<want: 187 < 199

@AlexanderYastrebov
Copy link
Member Author 

--- FAIL: TestFifo (3.27s)
    --- FAIL: TestFifo/fifo_with_reaching_max_concurrency_and_queue_timeouts (1.03s)
        fifo_test.go:260: aroute: * -> fifo(3, 5, "10ms") -> "http://127.0.0.1:42173"
        fifo_test.go:306: Success [0..1]: 0.10
        fifo_test.go:307: requests: 1935
        fifo_test.go:311: OK rate too low got<want: 192 < 194

@AlexanderYastrebov
Copy link
Member Author 

--- FAIL: TestFifo (3.26s)
    --- FAIL: TestFifo/fifo_simple_ok (1.01s)
        fifo_test.go:260: aroute: * -> fifo(3, 5, "1s") -> "http://127.0.0.1:41917"
        fifo_test.go:306: Success [0..1]: 0.99
        fifo_test.go:307: requests: 200
        fifo_test.go:311: OK rate too low got<want: 198 < 200

@AlexanderYastrebov
Copy link
Member Author 

--- FAIL: TestFifo (3.29s)
    --- FAIL: TestFifo/fifo_with_reaching_max_concurrency_and_queue_timeouts (1.04s)
        fifo_test.go:260: aroute: * -> fifo(3, 5, "10ms") -> "http://127.0.0.1:39415"
        fifo_test.go:306: Success [0..1]: 0.10
        fifo_test.go:307: requests: 1938
        fifo_test.go:311: OK rate too low got<want: 192 < 194

@AlexanderYastrebov
Copy link
Member Author 

--- FAIL: TestFifo (3.29s)
    --- FAIL: TestFifo/fifo_with_reaching_max_concurrency_and_queue_timeouts (1.03s)
        fifo_test.go:260: aroute: * -> fifo(3, 5, "10ms") -> "http://127.0.0.1:43297"
        fifo_test.go:306: Success [0..1]: 0.10
        fifo_test.go:307: requests: 1999
        fifo_test.go:311: OK rate too low got<want: 195 < 200

@szuecs
Copy link
Member

szuecs commented Oct 4, 2023 

2023/10/02 01:16:06 text reporter:
--- FAIL: TestFifo (3.28s)
    --- FAIL: TestFifo/fifo_with_reaching_max_concurrency_and_queue_timeouts (1.04s)
        fifo_test.go:260: aroute: * -> fifo(3, 5, "10ms") -> "http://127.0.0.1:36683/"
        fifo_test.go:306: Success [0..1]: 0.10
        fifo_test.go:307: requests: 1997
        fifo_test.go:311: OK rate too low got<want: 191 < 200

szuecs added a commit that referenced this issue Oct 4, 2023
@szuecs
fix: flakytest TestFifo/fifo_with_reaching_max_concurrency_and_queue_…
6f8ff52 
…timeouts #2557. The test is a little fuzzy so reduce the wanted ok rate by 0.0001 seems fine.

Signed-off-by: Sandor Szücs <sandor.szuecs@zalando.de>
@szuecs szuecs mentioned this issue Oct 4, 2023
Merged
szuecs added a commit that referenced this issue Oct 4, 2023
@szuecs
fix: flakytest TestFifo/fifo_with_reaching_max_concurrency_and_queue_…
41326c0 
…timeouts #2557. The test is a little fuzzy so reduce the wanted ok rate by 0.0001 seems fine. (#2644)

Signed-off-by: Sandor Szücs <sandor.szuecs@zalando.de>
@RomanZavodskikh
Copy link
Member 

--- FAIL: TestFifo (3.27s)
    --- FAIL: TestFifo/fifo_with_reaching_max_concurrency_and_queue_timeouts (1.02s)
        fifo_test.go:260: aroute: * -> fifo(3, 5, "10ms") -> "http://127.0.0.1:38387/"
        fifo_test.go:306: Success [0..1]: 0.10
        fifo_test.go:307: requests: 1926
        fifo_test.go:311: OK rate too low got<want: 184 < 193

https://github.com/zalando/skipper/actions/runs/6416414708/job/17420107022?pr=2648

@AlexanderYastrebov
Copy link
Member Author 

--- FAIL: TestFifo (3.29s)
    --- FAIL: TestFifo/fifo_with_reaching_max_concurrency_and_queue_timeouts (1.03s)
        fifo_test.go:260: aroute: * -> fifo(3, 5, "10ms") -> "http://127.0.0.1:44063"
        fifo_test.go:306: Success [0..1]: 0.10
        fifo_test.go:307: requests: 1991
        fifo_test.go:311: OK rate too low got<want: 199 < 199

@AlexanderYastrebov
Copy link
Member Author

@szuecs #2644 apparently did not help

@szuecs
Copy link
Member

szuecs commented Oct 5, 2023

Sounds like we need to weak it a bit more.
199 < 199 and 184 < 193 are close enough to tune this a little bit more

@szuecs szuecs mentioned this issue Oct 5, 2023
Merged
@RomanZavodskikh
Copy link
Member

https://github.com/zalando/skipper/actions/runs/6429586175/job/17458952714?pr=2634

--- FAIL: TestFifo (3.28s)
    --- FAIL: TestFifo/fifo_with_reaching_max_concurrency_and_queue_timeouts (1.03s)
        fifo_test.go:260: aroute: * -> fifo(3, 5, "10ms") -> "http://127.0.0.1:45907/"
        fifo_test.go:306: Success [0..1]: 0.10
        fifo_test.go:307: requests: 1992
        fifo_test.go:311: OK rate too low got<want: 198 < 199

@AlexanderYastrebov
Copy link
Member Author 

--- FAIL: TestFifo (3.30s)
    --- FAIL: TestFifo/fifo_with_reaching_max_concurrency_and_queue_timeouts (1.03s)
        fifo_test.go:260: aroute: * -> fifo(3, 5, "10ms") -> "http://127.0.0.1:45581"
        fifo_test.go:306: Success [0..1]: 0.10
        fifo_test.go:307: requests: 1912
        fifo_test.go:311: OK rate too low got<want: 190 < 191

@szuecs
Copy link
Member

szuecs commented Oct 31, 2023 

--- FAIL: TestFifo (3.29s)
    --- FAIL: TestFifo/fifo_with_reaching_max_concurrency_and_queue_timeouts (1.03s)
        fifo_test.go:260: aroute: * -> fifo(3, 5, "10ms") -> "http://127.0.0.1:45373/"
        fifo_test.go:306: Success [0..1]: 0.10
        fifo_test.go:307: requests: 1981
        fifo_test.go:311: OK rate too low got<want: 189 < 198

@szuecs
Copy link
Member

szuecs commented Nov 14, 2023 

--- FAIL: TestFifo (3.29s)
    --- FAIL: TestFifo/fifo_with_reaching_max_concurrency_and_queue_timeouts (1.03s)
        fifo_test.go:260: aroute: * -> fifo(3, 5, "10ms") -> "http://127.0.0.1:40309/"
        fifo_test.go:306: Success [0..1]: 0.10
        fifo_test.go:307: requests: 1901
        fifo_test.go:311: OK rate too low got<want: 189 < 190

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
flakytest
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: TestFifo requires more tuning zalando/skipper
3 participants
@szuecs @AlexanderYastrebov @RomanZavodskikh