Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: flaky validations tests #4012

Merged
merged 1 commit into from
Oct 26, 2023
Merged

fix: flaky validations tests #4012

merged 1 commit into from
Oct 26, 2023

Conversation

achettyiitr
Copy link
Member

@achettyiitr achettyiitr commented Oct 26, 2023

Description

  • Too many times, validation tests fail because we are consuming many docker containers, and the docker pool times out and gives EOF error.
https://github.com/rudderlabs/rudder-server/actions/runs/6650806396/job/18071584306?pr=4012
=== Failed
=== FAIL: warehouse/validations TestValidate (63.19s)
    validate_test.go:48: minio: localhost:32899
    validate_test.go:38: 
        	Error Trace:	/home/runner/work/rudder-server/rudder-server/warehouse/validations/validate_test.go:38
        	            				/home/runner/go/pkg/mod/golang.org/x/sync@v0.4.0/errgroup/errgroup.go:75
        	            				/opt/hostedtoolcache/go/1.21.3/x64/src/runtime/asm_amd64.s:1650
        	Error:      	Received unexpected error:
        	            	reached retry deadline: unexpected EOF
        	Test:       	TestValidate
    --- PASS: TestValidate/invalid_path (0.00s)
    --- PASS: TestValidate/steps (0.00s)
    --- PASS: TestValidate/validate (0.00s)
  • Sometimes we get a bucket already created error which should not happen since each test is running isolated. It seems like mix-match is happening when too many containers are being created.
https://github.com/rudderlabs/rudder-server/actions/runs/6650207486/job/18069945875
=== Failed
=== FAIL: warehouse/validations TestValidator/Fetch_schema (4.56s)
    validate_test.go:46: 
        	Error Trace:	/home/runner/work/rudder-server/rudder-server/warehouse/validations/validate_test.go:46
        	            				/home/runner/go/pkg/mod/golang.org/x/sync@v0.4.0/errgroup/errgroup.go:75
        	            				/opt/hostedtoolcache/go/1.21.3/x64/src/runtime/asm_amd64.s:1650
        	Error:      	Received unexpected error:
        	            	Your previous request to create the named bucket succeeded and you already own it.
        	Test:       	TestValidator/Fetch_schema
    validate_test.go:40: db: ***localhost:32898/jobsdb?sslmode=disable
    --- FAIL: TestValidator/Fetch_schema (4.56s)
  • We can intelligently use single docker for the validation tests since these are isolated with a namespace.

Linear Ticket

Security

  • The code changed/added as part of this pull request won't create any security issues with how the software is being used.

@achettyiitr achettyiitr changed the title fix: validations tests fix: flaky validations tests Oct 26, 2023
@fracasula fracasula merged commit 3b157e3 into master Oct 26, 2023
33 checks passed
@fracasula fracasula deleted the fix.validations-tests branch October 26, 2023 08:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants