Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CI unrelated failures #18077

Closed
wxiaoguang opened this issue Dec 23, 2021 · 4 comments · Fixed by #18447
Closed

CI unrelated failures #18077

wxiaoguang opened this issue Dec 23, 2021 · 4 comments · Fixed by #18447
Labels
type/bug type/testing

Comments

@wxiaoguang
Copy link
Contributor

During unit-test:

+ make unit-test-coverage test-check
Running unit-test-coverage -race -tags 'sqlite sqlite_unlock_notify'...
?   	code.gitea.io/gitea	[no test files]
ok  	code.gitea.io/gitea/build/codeformat	0.189s	coverage: 71.8% of statements
ok  	code.gitea.io/gitea/cmd	0.793s	coverage: 8.6% of statements
?   	code.gitea.io/gitea/contrib/environment-to-ini	[no test files]
......
?   	code.gitea.io/gitea/services/task	[no test files]
ok  	code.gitea.io/gitea/services/user	6.467s	coverage: 30.7% of statements
ok  	code.gitea.io/gitea/services/webhook	4.276s	coverage: 62.0% of statements
ok  	code.gitea.io/gitea/services/wiki	13.117s	coverage: 65.5% of statements

==>�Ok

Running test-check...
make test-backend has changed files in the source tree:
?? mssql-log/
?? mysql-log/
?? mysql8-log/
You should change the tests to create these files in a temporary directory.
Do not simply add these files to .gitignore
make: *** [Makefile:368: test-check] Error 1
@wxiaoguang wxiaoguang mentioned this issue Jan 29, 2022
Closed
@zeripath
Copy link
Contributor

I think this indicates that you've ran some other tests (Likely the integration tests make test-mssql etc.) which have created those log files.

The purpose of the make test-check is to detect if test-backend has created any files only and it assumes that there are not any uncommited files at the start of the test. It's not trying to be clever.

Therefore we have four solutions:

  • Change the wording here to make it less certain and say there are uncommitted files - if these were present before you ran test-backend there's nothing to worry about but otherwise you need to find out what has happened.
  • Add these to .gitignore
  • Change the integrations tests to make sure that they put their logs with the rest of their data, or put them in a new directory e.g. integrations-test-logs which can be ignored.
  • Make the test a bit cleverer so it only detects newly changed files.

I think however as this is only a rough and ready test to detect if test-backend is doing naughty things on drone I'm not sure any of this is worth doing.

@wxiaoguang
Copy link
Contributor Author

wxiaoguang commented Jan 29, 2022 •
edited
Loading

Hmm ... it happens during our CI, and only happens randomly, makes some PRs fails again and again. I haven't caught the reason or investigated.

Among these solutions, which one is best?

@zeripath
Copy link
Contributor

oh that's interesting.

I guess we should change the drone configuration to make sure that these logs end up on the temp dir.

I'll take a look at this.

zeripath added a commit to zeripath/gitea that referenced this issue Jan 29, 2022
@zeripath
Change log root for integration tests to use the REPO_TEST_DIR
21eb803 
There is a potential race in the drone integration tests whereby test-mysql etc
will start writing to log files causing make test-check fail.

Fix go-gitea#18077

Signed-off-by: Andrew Thornton <art27@cantab.net>
@zeripath zeripath mentioned this issue Jan 29, 2022
Merged
@zeripath
Copy link
Contributor

I've folded the fix in to #18447

zeripath added a commit that referenced this issue Jan 29, 2022
@zeripath
Ensure git tag tests and others create test repos in tmpdir (#18447)
f7b152f 
* Ensure git tag tests and other create test repos in tmpdir

There are a few places where tests appear to reuse testing repos which
causes random CI failures.

This PR simply changes these tests to ensure that cloning always happens
into new temporary directories.

Fix #18444

* Change log root for integration tests to use the REPO_TEST_DIR

There is a potential race in the drone integration tests whereby test-mysql etc
will start writing to log files causing make test-check fail.

Fix #18077

Signed-off-by: Andrew Thornton <art27@cantab.net>
@zeripath zeripath mentioned this issue Feb 5, 2022
Merged
zeripath added a commit to zeripath/gitea that referenced this issue Feb 14, 2022
@zeripath
Ensure git tag tests and others create test repos in tmpdir (go-gitea…
9cf32ba 
…#18447)

Backport go-gitea#18447

* Ensure git tag tests and other create test repos in tmpdir

There are a few places where tests appear to reuse testing repos which
causes random CI failures.

This PR simply changes these tests to ensure that cloning always happens
into new temporary directories.

Fix go-gitea#18444

* Change log root for integration tests to use the REPO_TEST_DIR

There is a potential race in the drone integration tests whereby test-mysql etc
will start writing to log files causing make test-check fail.

Fix go-gitea#18077

Signed-off-by: Andrew Thornton <art27@cantab.net>
@zeripath zeripath mentioned this issue Feb 14, 2022
Merged
lunny pushed a commit that referenced this issue Feb 15, 2022
@zeripath @6543
Ensure git tag tests and others create test repos in tmpdir (#18447) (#…
c876124 
…18767)

Backport #18447

* Ensure git tag tests and other create test repos in tmpdir

There are a few places where tests appear to reuse testing repos which
causes random CI failures.

This PR simply changes these tests to ensure that cloning always happens
into new temporary directories.

Fix #18444

* Change log root for integration tests to use the REPO_TEST_DIR

There is a potential race in the drone integration tests whereby test-mysql etc
will start writing to log files causing make test-check fail.

Fix #18077

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
Chianina pushed a commit to Chianina/gitea that referenced this issue Mar 28, 2022
@zeripath
Ensure git tag tests and others create test repos in tmpdir (go-gitea…
822be8e 
…#18447)

* Ensure git tag tests and other create test repos in tmpdir

There are a few places where tests appear to reuse testing repos which
causes random CI failures.

This PR simply changes these tests to ensure that cloning always happens
into new temporary directories.

Fix go-gitea#18444

* Change log root for integration tests to use the REPO_TEST_DIR

There is a potential race in the drone integration tests whereby test-mysql etc
will start writing to log files causing make test-check fail.

Fix go-gitea#18077

Signed-off-by: Andrew Thornton <art27@cantab.net>
@go-gitea go-gitea locked and limited conversation to collaborators Apr 28, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
type/bug type/testing
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Ensure git tag tests and others create test repos in tmpdir zeripath/gitea
3 participants
@silverwind @zeripath @wxiaoguang