Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Flaky tests on AppVeyor #5393

Closed
quasicomputational opened this issue Jun 22, 2018 · 3 comments
Closed

Flaky tests on AppVeyor #5393

quasicomputational opened this issue Jun 22, 2018 · 3 comments

Comments

@quasicomputational
Copy link
Contributor

Here's a typical example, from #5392:

PackageTests\CustomDep\cabal.test.hs                                                                              FAIL (59.84s)
$ "C:\ProgramData\chocolatey\lib\ghc\tools\ghc-8.0.2\bin\runghc.exe" "--" "--ghc-arg=-i" "--ghc-arg=-package-id" "--ghc-arg=Cabal-2.3.0.0-DOb3lzXR40i24avrU9Cnkv" "--ghc-arg=-package-id" "--ghc-arg=async-2.2.1-FtyEBgS0La811OtEcI8hQO" "--ghc-arg=-package-id" "--ghc-arg=base-4.9.1.0" "--ghc-arg=-package-id" "--ghc-arg=cabal-testsuite-2.3.0.0-KvRMCPAcFo839elZjrhH09" "--ghc-arg=-package-id" "--ghc-arg=clock-0.7.2-2aZP57zFattJce1nK6pmuj" "--ghc-arg=-package-id" "--ghc-arg=exceptions-0.10.0-D3l6tOLudZU9YCvtoPHr4p" "--ghc-arg=-package-id" "--ghc-arg=filepath-1.4.1.1" "--ghc-arg=-package-id" "--ghc-arg=optparse-applicative-0.14.2.0-8qLJq3LaLRK948X9OctCJT" "--ghc-arg=-package-id" "--ghc-arg=process-1.4.3.0" "--ghc-arg=-package-id" "--ghc-arg=transformers-0.5.2.0" "PackageTests\CustomDep\cabal.test.hs" "--builddir" "C:\projects\cabal\cabal-testsuite\dist" "PackageTests\CustomDep\cabal.test.hs" "--with-cabal" "..\cabal-install\dist\build\cabal\cabal.exe" "--skip-setup-tests"
stdout:
+ "C:\projects\cabal\cabal-testsuite\..\cabal-install\dist\build\cabal\cabal.exe" "new-build" "-vverbose +markoutput +nowrap" "--builddir" "C:\projects\cabal\cabal-testsuite\PackageTests\CustomDep\cabal.dist\work\.\dist" "--project-file" "cabal.project" "-j1" "all"
Project settings changed, reconfiguring...
creating C:\projects\cabal\cabal-testsuite\PackageTests\CustomDep\cabal.dist\work\dist
creating C:\projects\cabal\cabal-testsuite\PackageTests\CustomDep\cabal.dist\work
creating C:\projects\cabal\cabal-testsuite\PackageTests\CustomDep\cabal.dist\work\dist
creating C:\projects\cabal\cabal-testsuite\PackageTests\CustomDep\cabal.dist\work\dist\cache
this build was affected by the following (project) config files:
- C:\projects\cabal\cabal-testsuite\PackageTests\CustomDep\cabal.project
Compiler settings changed, reconfiguring...
"C:\ProgramData\chocolatey\lib\ghc\tools\ghc-8.0.2\bin\ghc.exe" "--numeric-version"
looking for tool ghc-pkg near compiler in C:\ProgramData\chocolatey\lib\ghc\tools\ghc-8.0.2\bin
found ghc-pkg in C:\ProgramData\chocolatey\lib\ghc\tools\ghc-8.0.2\bin\ghc-pkg.exe
"C:\ProgramData\chocolatey\lib\ghc\tools\ghc-8.0.2\bin\ghc-pkg.exe" "--version"
"C:\ProgramData\chocolatey\lib\ghc\tools\ghc-8.0.2\bin\ghc.exe" "--supported-languages"
"C:\ProgramData\chocolatey\lib\ghc\tools\ghc-8.0.2\bin\ghc.exe" "--info"
"C:\ProgramData\chocolatey\lib\ghc\tools\ghc-8.0.2\bin\ghc.exe" "--print-global-package-db"
Reading available packages of hackage.haskell.org...
Using most recent state specified from most recent cabal update
Updating index cache file C:\Users\appveyor\AppData\Roaming\cabal\packages\hackage.haskell.org\01-index.cache ...
Index cache updated to index-state 2018-06-22T10:42:40Z
index-state(hackage.haskell.org) = 2018-06-22T10:42:40Z
Failed to query pkg-config, Cabal will continue without solving for pkg-config constraints: dieVerbatim: user error (CallStack (from HasCallStack):
  die', called at .\Distribution\Simple\Program\Db.hs:423:28 in Cabal-2.3.0.0-DOb3lzXR40i24avrU9Cnkv:Distribution.Simple.Program.Db
  requireProgram, called at .\Distribution\Solver\Types\PkgConfigDb.hs:65:21 in cabal-install-2.3.0.0-GMgA89eWLTXFbpy6Kjhfis:Distribution.Solver.Types.PkgConfigDb
cabal.exe: The program 'pkg-config' is required but it could not be found.
)
-----BEGIN CABAL OUTPUT-----
Resolving dependencies...
-----END CABAL OUTPUT-----
Component graph for old-locale-1.0.0.7: component lib
component old-locale-1.0.0.7-0daa8146066c73a04b741a3c8264dbfb542f4843
    include base-4.9.1.0
unit old-locale-1.0.0.7-0daa8146066c73a04b741a3c8264dbfb542f4843
    include base-4.9.1.0
    System.Locale=old-locale-1.0.0.7-0daa8146066c73a04b741a3c8264dbfb542f4843:System.Locale
Component graph for old-time-1.1.0.3: component lib
component old-time-1.1.0.3-3b1f7c1e6e78f38c7b5483ed825e9e941c195086
    include base-4.9.1.0
    include old-locale-1.0.0.7-0daa8146066c73a04b741a3c8264dbfb542f4843
unit old-time-1.1.0.3-3b1f7c1e6e78f38c7b5483ed825e9e941c195086
    include base-4.9.1.0
    include old-locale-1.0.0.7-0daa8146066c73a04b741a3c8264dbfb542f4843
    System.Time=old-time-1.1.0.3-3b1f7c1e6e78f38c7b5483ed825e9e941c195086:System.Time
Component graph for custom-0.1.0.0: component lib
component custom-0.1.0.0-inplace include base-4.9.1.0
unit custom-0.1.0.0-inplace
    include base-4.9.1.0
    A=custom-0.1.0.0-inplace:A
Component graph for client-0.1.0.0: component lib
component client-0.1.0.0-inplace
    include base-4.9.1.0
    include custom-0.1.0.0-inplace
unit client-0.1.0.0-inplace
    include base-4.9.1.0
    include custom-0.1.0.0-inplace
    B=client-0.1.0.0-inplace:B
-----BEGIN CABAL OUTPUT-----
Build profile: -w ghc-8.0.2 -O1
In order, the following will be built:
 - old-locale-1.0.0.7 (lib) (requires download & build)
 - old-time-1.1.0.3 (lib:old-time) (requires download & build)
 - custom-0.1.0.0 (lib:custom) (first run)
 - client-0.1.0.0 (lib) (first run)
-----END CABAL OUTPUT-----
creating C:\projects\cabal\cabal-testsuite\PackageTests\CustomDep\cabal.dist\work\dist\build
creating C:\projects\cabal\cabal-testsuite\PackageTests\CustomDep\cabal.dist\work\dist\tmp
creating C:\Users\appveyor\AppData\Roaming\cabal\store\ghc-8.0.2
"C:\ProgramData\chocolatey\lib\ghc\tools\ghc-8.0.2\bin\ghc-pkg.exe" "init" "C:\Users\appveyor\AppData\Roaming\cabal\store\ghc-8.0.2\package.db"
ghc-pkg.exe: cannot create: C:\Users\appveyor\AppData\Roaming\cabal\store\ghc-8.0.2\package.db already exists
stderr:
*** Exception: Command "C:\projects\cabal\cabal-testsuite\..\cabal-install\dist\build\cabal\cabal.exe" "new-build" "-vverbose +markoutput +nowrap" "--builddir" "C:\projects\cabal\cabal-testsuite\PackageTests\CustomDep\cabal.dist\work\.\dist" "--project-file" "cabal.project" "-j1" "all" failed.
Output:
Project settings changed, reconfiguring...
creating C:\projects\cabal\cabal-testsuite\PackageTests\CustomDep\cabal.dist\work\dist
creating C:\projects\cabal\cabal-testsuite\PackageTests\CustomDep\cabal.dist\work
creating C:\projects\cabal\cabal-testsuite\PackageTests\CustomDep\cabal.dist\work\dist
creating C:\projects\cabal\cabal-testsuite\PackageTests\CustomDep\cabal.dist\work\dist\cache
this build was affected by the following (project) config files:
- C:\projects\cabal\cabal-testsuite\PackageTests\CustomDep\cabal.project
Compiler settings changed, reconfiguring...
"C:\ProgramData\chocolatey\lib\ghc\tools\ghc-8.0.2\bin\ghc.exe" "--numeric-version"
looking for tool ghc-pkg near compiler in C:\ProgramData\chocolatey\lib\ghc\tools\ghc-8.0.2\bin
found ghc-pkg in C:\ProgramData\chocolatey\lib\ghc\tools\ghc-8.0.2\bin\ghc-pkg.exe
"C:\ProgramData\chocolatey\lib\ghc\tools\ghc-8.0.2\bin\ghc-pkg.exe" "--version"
"C:\ProgramData\chocolatey\lib\ghc\tools\ghc-8.0.2\bin\ghc.exe" "--supported-languages"
"C:\ProgramData\chocolatey\lib\ghc\tools\ghc-8.0.2\bin\ghc.exe" "--info"
"C:\ProgramData\chocolatey\lib\ghc\tools\ghc-8.0.2\bin\ghc.exe" "--print-global-package-db"
Reading available packages of hackage.haskell.org...
Using most recent state specified from most recent cabal update
Updating index cache file C:\Users\appveyor\AppData\Roaming\cabal\packages\hackage.haskell.org\01-index.cache ...
Index cache updated to index-state 2018-06-22T10:42:40Z
index-state(hackage.haskell.org) = 2018-06-22T10:42:40Z
Failed to query pkg-config, Cabal will continue without solving for pkg-config constraints: dieVerbatim: user error (CallStack (from HasCallStack):
  die', called at .\Distribution\Simple\Program\Db.hs:423:28 in Cabal-2.3.0.0-DOb3lzXR40i24avrU9Cnkv:Distribution.Simple.Program.Db
  requireProgram, called at .\Distribution\Solver\Types\PkgConfigDb.hs:65:21 in cabal-install-2.3.0.0-GMgA89eWLTXFbpy6Kjhfis:Distribution.Solver.Types.PkgConfigDb
cabal.exe: The program 'pkg-config' is required but it could not be found.
)
-----BEGIN CABAL OUTPUT-----
Resolving dependencies...
-----END CABAL OUTPUT-----
Component graph for old-locale-1.0.0.7: component lib
component old-locale-1.0.0.7-0daa8146066c73a04b741a3c8264dbfb542f4843
    include base-4.9.1.0
unit old-locale-1.0.0.7-0daa8146066c73a04b741a3c8264dbfb542f4843
    include base-4.9.1.0
    System.Locale=old-locale-1.0.0.7-0daa8146066c73a04b741a3c8264dbfb542f4843:System.Locale
Component graph for old-time-1.1.0.3: component lib
component old-time-1.1.0.3-3b1f7c1e6e78f38c7b5483ed825e9e941c195086
    include base-4.9.1.0
    include old-locale-1.0.0.7-0daa8146066c73a04b741a3c8264dbfb542f4843
unit old-time-1.1.0.3-3b1f7c1e6e78f38c7b5483ed825e9e941c195086
    include base-4.9.1.0
    include old-locale-1.0.0.7-0daa8146066c73a04b741a3c8264dbfb542f4843
    System.Time=old-time-1.1.0.3-3b1f7c1e6e78f38c7b5483ed825e9e941c195086:System.Time
Component graph for custom-0.1.0.0: component lib
component custom-0.1.0.0-inplace include base-4.9.1.0
unit custom-0.1.0.0-inplace
    include base-4.9.1.0
    A=custom-0.1.0.0-inplace:A
Component graph for client-0.1.0.0: component lib
component client-0.1.0.0-inplace
    include base-4.9.1.0
    include custom-0.1.0.0-inplace
unit client-0.1.0.0-inplace
    include base-4.9.1.0
    include custom-0.1.0.0-inplace
    B=client-0.1.0.0-inplace:B
-----BEGIN CABAL OUTPUT-----
Build profile: -w ghc-8.0.2 -O1
In order, the following will be built:
 - old-locale-1.0.0.7 (lib) (requires download & build)
 - old-time-1.1.0.3 (lib:old-time) (requires download & build)
 - custom-0.1.0.0 (lib:custom) (first run)
 - client-0.1.0.0 (lib) (first run)
-----END CABAL OUTPUT-----
creating C:\projects\cabal\cabal-testsuite\PackageTests\CustomDep\cabal.dist\work\dist\build
creating C:\projects\cabal\cabal-testsuite\PackageTests\CustomDep\cabal.dist\work\dist\tmp
creating C:\Users\appveyor\AppData\Roaming\cabal\store\ghc-8.0.2
"C:\ProgramData\chocolatey\lib\ghc\tools\ghc-8.0.2\bin\ghc-pkg.exe" "init" "C:\Users\appveyor\AppData\Roaming\cabal\store\ghc-8.0.2\package.db"
ghc-pkg.exe: cannot create: C:\Users\appveyor\AppData\Roaming\cabal\store\ghc-8.0.2\package.db already exists
*** unexpected failure for PackageTests\CustomDep\cabal.test.hs

These failures aren't happening on every attempt; maybe half or a quarter are affected? I've never seen anything like that from Travis, so it looks like it might be Windows-only, or something to do with the specific AppVeyor environment.
@quasicomputational
Copy link
Contributor Author

#4005 is another AppVeyor thing apparently caused by concurrent access to the package database, and fixed in 8.2. Maybe this is related and it would go away if we upgraded AppVeyor's GHC?

@hvr
Copy link
Member

hvr commented Jun 23, 2018

@quasicomputational there's one way to find out, isn't there? :)

quasicomputational added a commit to quasicomputational/cabal that referenced this issue Jun 24, 2018
@quasicomputational
Bump AppVeyor's GHC to 8.4.3
b4acff1 
In the hope that this will solve haskell#5393.
@quasicomputational quasicomputational mentioned this issue Jun 24, 2018
Closed
4 tasks
quasicomputational added a commit to quasicomputational/cabal that referenced this issue Jun 24, 2018
@quasicomputational
Bump AppVeyor's GHC to 8.4.3
20411e4 
In the hope that this will solve haskell#5393.
@phadej
Copy link
Collaborator

phadej commented Jan 20, 2020

appveyor have been quite reliable.

@phadej phadej closed this as completed Jan 20, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@phadej @hvr @quasicomputational