testing: TempDir RemoveAll cleanup failures with "The process cannot access the file because it is being used by another process." #51442

bcmills · 2022-03-02T20:25:53Z

--- FAIL: TestRWithNonexistentFile (0.83s)
    testing.go:1090: TempDir RemoveAll cleanup: remove C:\Users\gopher\AppData\Local\Temp\1\TestRWithNonexistentFile1599077814\001\pack.exe: The process cannot access the file because it is being used by another process.
FAIL
FAIL	cmd/pack	4.916s

This can probably be fixed by expanding the set of errors handled by the retry loop added for #50051.

greplogs --dashboard -md -l -e 'TempDir RemoveAll cleanup: .* The process cannot access the file because it is being used by another process\.'

The text was updated successfully, but these errors were encountered:

bcmills · 2022-03-02T20:51:49Z

(This error message is 0x20 ERROR_SHARING_VIOLATION.)

gopherbot · 2022-03-02T21:38:01Z

Change https://go.dev/cl/388916 mentions this issue: testing: include ERROR_SHARING_VIOLATION in Windows cleanup retries

bhasden · 2022-05-18T20:50:27Z

Apologies for commenting on a closed issue. However, I wanted to ask if there is any chance this makes its way out prior to 1.19?

ianlancetaylor · 2022-05-18T20:56:29Z

@bhasden Are you encountering this yourself? What Go version and what GOOS/GOARCH? Thanks.

bcmills · 2022-05-18T21:04:40Z

FWIW, this fix would be easy to backport to Go 1.18, but somewhat less trivial to backport to Go 1.17 (it would also need CL 371296, which is a bit more invasive).

There is also a workaround that you can apply for your own specific tests independent of the Go version: you can defer (or t.Cleanup) your own function that iterates os.RemoveAll on the directory returned by TempDir until it is successfully removed.

bhasden · 2022-05-18T21:48:51Z

@bhasden Are you encountering this yourself? What Go version and what GOOS/GOARCH? Thanks.

I believe I'm running into this same issue in Golang 1.18.1 and 1.18.2 (only two versions I've tested) with some casbin tests that are loading a model configuration file. It's only reproducible in Windows. Team members on non-Windows machines are not running into the issue.

go env output:

set GO111MODULE=on
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\BrianHasden\AppData\Local\go-build
set GOENV=C:\Users\BrianHasden\AppData\Roaming\go\env
set GOEXE=.exe
set GOEXPERIMENT=
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\BrianHasden\go\pkg\mod
set GONOPROXY=######
set GONOSUMDB=######
set GOOS=windows
set GOPATH=C:\Users\BrianHasden\go
set GOPRIVATE=######
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=C:\Users\BrianHasden\go\go1.18.2
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=C:\Users\BrianHasden\go\go1.18.2\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=go1.18.2
set GCCGO=gccgo
set GOAMD64=v1
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=######
set GOWORK=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\BRIANH~1\AppData\Local\Temp\go-build2379612813=/tmp/go-build -gno-record-gcc-switches

ianlancetaylor · 2022-05-19T00:07:42Z

@gopherbot Please open backport to 1.18.

At least one person is encountering this problem in 1.18. There are workarounds but they are awkward. The fix would be harder to backport to earlier releases.

gopherbot · 2022-05-19T00:07:56Z

Backport issue(s) opened: #52986 (for 1.18).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://go.dev/wiki/MinorReleases.

gopherbot · 2022-05-23T16:21:03Z

Change https://go.dev/cl/407877 mentions this issue: [release-branch.go1.18] testing: include ERROR_SHARING_VIOLATION in Windows cleanup retries

…indows cleanup retries Fixes #52986 Updates #51442 Updates #50051 Change-Id: I1bfbc08c907077467fd50febbec6299a9b73af41 Reviewed-on: https://go-review.googlesource.com/c/go/+/388916 Trust: Bryan Mills <bcmills@google.com> Run-TryBot: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> (cherry picked from commit eeb9f09) Reviewed-on: https://go-review.googlesource.com/c/go/+/407877 Reviewed-by: Nooras Saba‎ <saba@golang.org> Auto-Submit: Bryan Mills <bcmills@google.com>

… windows bug fix related to the following issue: golang/go#51442

i4ki · 2023-04-13T23:12:30Z

I'm still experiencing this issue in go1.20.3.

There is also a workaround that you can apply for your own specific tests independent of the Go version: you can defer (or t.Cleanup) your own function that iterates os.RemoveAll on the directory returned by TempDir until it is successfully removed.

@bcmills this workaround still works.

It looks like the 2 seconds timeout in the removeAll workaround of the testing package is not enough...
https://cs.opensource.google/go/go/+/master:src/testing/testing.go;l=1248-1276;drc=master

ianlancetaylor · 2023-04-13T23:14:41Z

@i4ki This issue has been closed for almost a year, and we think it is fixed. If you are seeing a problem with 1.20, please open a new issue with reproduction details. It may be the same problem, but it may be a new one. Thanks.

rasa · 2023-05-04T04:48:10Z

I'm experiencing the same error in syncthing's test suite on go 1.20.3:

go test -r TestHandleFile ./...
...
--- FAIL: TestHandleFile (1.60s)
    testing.go:1206: TempDir RemoveAll cleanup: remove C:\Users\ross\AppData\Local\Temp\TestHandleFile1557195908\001: The process cannot access the file because it is being used by another process.
FAIL
FAIL    github.com/syncthing/syncthing/lib/model        1.821s
...
--- FAIL: TestHandleFileWithTemp (1.45s)
    testing.go:1206: TempDir RemoveAll cleanup: remove C:\Users\ross\AppData\Local\Temp\TestHandleFileWithTemp2880978547\001: The process cannot access the file because it is being used by another process.
FAIL
FAIL    github.com/syncthing/syncthing/lib/model        1.578s

rasa · 2023-05-04T15:20:32Z

Syncthing is solving this by rewriting the code to use an emulated file system in syncthing/syncthing#8889

bcmills · 2023-05-04T15:32:36Z

@rasa, as far as we know the retry loop that was added for this issue works.
It is likely that your test is actually leaking an open file when the test returns.
If you can definitely rule that out, please file a new issue.

bcmills added NeedsFix The path to resolution is known, but the work has not been done. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Mar 2, 2022

bcmills added this to the Go1.19 milestone Mar 2, 2022

bcmills self-assigned this Mar 2, 2022

gopherbot removed the NeedsFix The path to resolution is known, but the work has not been done. label Mar 2, 2022

bcmills added NeedsFix The path to resolution is known, but the work has not been done. release-blocker labels Mar 2, 2022

gopherbot removed the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Mar 2, 2022

bcmills added the OS-Windows label Mar 2, 2022

gopherbot closed this as completed in eeb9f09 Mar 3, 2022

znewman01 mentioned this issue Apr 24, 2022

Allow passing keys via environment variables (env:// refs) sigstore/cosign#1794

Merged

gopherbot mentioned this issue May 19, 2022

testing: TempDir RemoveAll cleanup failures with "The process cannot access the file because it is being used by another process." [1.18 backport] #52986

Closed

jLopezbarb mentioned this issue May 26, 2022

Revert refactor unit test folders and envs (#2713) okteto/okteto#2725

Merged

rsc unassigned bcmills Jun 22, 2022

bcmills mentioned this issue Jul 12, 2022

x/tools/gopls: test cleanup errors during shutdown with ERROR_SHARING_VIOLATION on Windows #53819

Closed

weikanglim mentioned this issue Sep 2, 2022

[CI/CD Issues] - TempDir RemoveAll cleanup fails Azure/azure-dev#580

Closed

nao1215 added a commit to nao1215/gup that referenced this issue Sep 18, 2022

Fixed code to avoid "golang/go#51442" bug

f524fc6

vaikas mentioned this issue Sep 26, 2022

feat: Add Filesystem based remote store to support airgap. theupdateframework/go-tuf#397

Merged

2 tasks

haoel mentioned this issue Oct 11, 2022

using go 1.19 to run unit test megaease/easeprobe#227

Merged

haoel mentioned this issue Oct 11, 2022

using retry loop to remove the temp dir megaease/easeprobe#228

Merged

pnasrat mentioned this issue Dec 1, 2022

Fix file URLs with digest lima-vm/lima#1219

Merged

imsodin mentioned this issue Feb 8, 2023

Test cleanup fails all model tests on Windows on Go 1.20 syncthing/syncthing#8779

Closed

YrrepNoj added a commit to zarf-dev/zarf that referenced this issue Mar 13, 2023

retry removals of directories that have been recently archived b/c of…

c8670f1

… windows bug fix related to the following issue: golang/go#51442

bhapas mentioned this issue Apr 22, 2023

x-pack/filebeat : Sanitize cel input resource trace filename elastic/beats#35154

Merged

6 tasks

golang locked as resolved and limited conversation to collaborators May 4, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

testing: TempDir RemoveAll cleanup failures with "The process cannot access the file because it is being used by another process." #51442

testing: TempDir RemoveAll cleanup failures with "The process cannot access the file because it is being used by another process." #51442

bcmills commented Mar 2, 2022

bcmills commented Mar 2, 2022

gopherbot commented Mar 2, 2022

bhasden commented May 18, 2022

ianlancetaylor commented May 18, 2022

bcmills commented May 18, 2022

bhasden commented May 18, 2022 •

edited

Loading

ianlancetaylor commented May 19, 2022

gopherbot commented May 19, 2022

gopherbot commented May 23, 2022

i4ki commented Apr 13, 2023

ianlancetaylor commented Apr 13, 2023

rasa commented May 4, 2023

rasa commented May 4, 2023

bcmills commented May 4, 2023

testing: TempDir RemoveAll cleanup failures with "The process cannot access the file because it is being used by another process." #51442

testing: TempDir RemoveAll cleanup failures with "The process cannot access the file because it is being used by another process." #51442

Comments

bcmills commented Mar 2, 2022

bcmills commented Mar 2, 2022

gopherbot commented Mar 2, 2022

bhasden commented May 18, 2022

ianlancetaylor commented May 18, 2022

bcmills commented May 18, 2022

bhasden commented May 18, 2022 • edited Loading

ianlancetaylor commented May 19, 2022

gopherbot commented May 19, 2022

gopherbot commented May 23, 2022

i4ki commented Apr 13, 2023

ianlancetaylor commented Apr 13, 2023

rasa commented May 4, 2023

rasa commented May 4, 2023

bcmills commented May 4, 2023

bhasden commented May 18, 2022 •

edited

Loading