Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Doctests in multi-package project interfering with each other #5159

Closed
sjakobi opened this issue Feb 3, 2020 · 2 comments
Closed

Doctests in multi-package project interfering with each other #5159

sjakobi opened this issue Feb 3, 2020 · 2 comments

Comments

@sjakobi
Copy link
Member

sjakobi commented Feb 3, 2020 •
edited
Loading

General summary/comments (optional)

In a multi-package project that contains several doctest suites, stack test fails with a "could not load module from hidden package" message. Only when I add -j1, the tests pass reliably.

It seems that the package environment contains the dependencies for one package, but the other package tries to use it too, which fails because the second package has additional dependencies.

Steps to reproduce

$ git clone https://github.com/quchen/prettyprinter
$ cd prettyprinter
$ git checkout 320538be76cff615c44336b84f28e88d42988830
$ stack test --no-run-tests
$ stack test prettyprinter:doctest prettyprinter-convert-ansi-wl-pprint:doctest

Expected

Both testsuites pass.

Actual

$ stack test prettyprinter:doctest prettyprinter-convert-ansi-wl-pprint:doctest
prettyprinter                       > test (suite: doctest)
prettyprinter-convert-ansi-wl-pprint> test (suite: doctest)
            

prettyprinter-convert-ansi-wl-pprint> Loaded package environment from /tmp/stack32735/test-ghc-env
prettyprinter                       > Loaded package environment from /tmp/stack32735/test-ghc-env
prettyprinter-convert-ansi-wl-pprint> Loaded package environment from /tmp/stack32735/test-ghc-env
prettyprinter                       > Loaded package environment from /tmp/stack32735/test-ghc-env
prettyprinter-convert-ansi-wl-pprint> Loaded package environment from /tmp/stack32735/test-ghc-env
prettyprinter-convert-ansi-wl-pprint> Loaded package environment from /tmp/stack32735/test-ghc-env
prettyprinter-convert-ansi-wl-pprint> Examples: 0  Tried: 0  Errors: 0  Failures: 0
prettyprinter-convert-ansi-wl-pprint> Test suite doctest passed
prettyprinter                       > Loaded package environment from /tmp/stack32735/test-ghc-env
prettyprinter                       > Loaded package environment from /tmp/stack32735/test-ghc-env
prettyprinter                       > src/Data/Text/Prettyprint/Doc/Internal.hs:1948: failure in expression `import Test.QuickCheck.Modifiers'
prettyprinter                       > expected: 
prettyprinter                       >  but got: 
prettyprinter                       >           <no location info>: error:
prettyprinter                       >               Could not load module ‘Test.QuickCheck.Modifiers’
prettyprinter                       >               It is a member of the hidden package ‘QuickCheck-2.12.6.1’.
prettyprinter                       >               You can run ‘:set -package QuickCheck’ to expose it.
prettyprinter                       >               (Note: this unloads all the modules in the current scope.)
prettyprinter                       > 
prettyprinter                       > Examples: 249  Tried: 103  Errors: 0  Failures: 1
prettyprinter                       > Test suite doctest failed
Completed 2 action(s).     
Test suite failure for package prettyprinter-1.6.0
    doctest:  exited with: ExitFailure 1
Logs printed to console

With --verbose:

$ stack test prettyprinter:doctest prettyprinter-convert-ansi-wl-pprint:doctest --verbose
Version 2.1.3, Git revision 636e3a759d51127df2b62f90772def126cdf6d1f (7735 commits) x86_64 hpack-0.31.2
2020-02-03 15:51:18.309701: [debug] Checking for project config at: /home/simon/src/prettyprinter/stack.yaml
2020-02-03 15:51:18.309832: [debug] Loading project config file stack.yaml
2020-02-03 15:51:18.313093: [debug] SELECT COUNT(*) FROM "last_performed" WHERE ("action"=?) AND ("timestamp">=?); [PersistInt64 1,PersistUTCTime 2020-02-02 14:51:18.313063578 UTC]
2020-02-03 15:51:18.313527: [debug] Using package location completions from a lock file
2020-02-03 15:51:18.449539: [debug] Asking for a supported GHC version
2020-02-03 15:51:18.449837: [debug] Installed tools: 
 - ghc-8.6.5
 - ghc-8.2.2
 - ghc-7.8.4
 - ghc-8.4.4
 - ghc-8.8.1
 - ghc-7.10.3
 - ghc-8.8.2
 - ghc-8.0.2
2020-02-03 15:51:18.450338: [debug] Run process: /sbin/ldconfig -p
2020-02-03 15:51:18.452089: [debug] Process finished in 2ms: /sbin/ldconfig -p
2020-02-03 15:51:18.452236: [debug] Found shared library libtinfo.so.5 in 'ldconfig -p' output
2020-02-03 15:51:18.452555: [debug] Did not find shared library libtinfo.so.6
2020-02-03 15:51:18.452622: [debug] Did not find shared library libncursesw.so.6
2020-02-03 15:51:18.452668: [debug] Found shared library libgmp.so.10 in 'ldconfig -p' output
2020-02-03 15:51:18.452730: [debug] Did not find shared library libgmp.so.3
2020-02-03 15:51:18.452754: [debug] Potential GHC builds: standard
2020-02-03 15:51:18.452780: [debug] Found already installed GHC builds: standard
2020-02-03 15:51:18.452949: [debug] SELECT "id","actual_version","arch","ghc_path","ghc_size","ghc_modified","ghc_pkg_path","runghc_path","haddock_path","cabal_version","global_db","global_db_cache_size","global_db_cache_modified","info","global_dump" FROM "compiler_cache" WHERE "ghc_path"=?; [PersistText "/home/simon/.stack/programs/x86_64-linux/ghc-8.6.5/bin/ghc-8.6.5"]
2020-02-03 15:51:18.500427: [debug] Loaded compiler information from cache
2020-02-03 15:51:18.500550: [debug] Asking for a supported GHC version
2020-02-03 15:51:18.500845: [debug] Resolving package entries
2020-02-03 15:51:18.500895: [debug] Parsing the targets
2020-02-03 15:51:18.512562: [debug] Checking flags
2020-02-03 15:51:18.512702: [debug] SourceMap constructed
2020-02-03 15:51:18.517107: [debug] Starting to execute command inside EnvConfig
2020-02-03 15:51:18.520818: [debug] Finding out which packages are already installed
2020-02-03 15:51:18.520949: [debug] Run process: /home/simon/.stack/programs/x86_64-linux/ghc-8.6.5/bin/ghc-pkg-8.6.5 --global --no-user-package-db dump --expand-pkgroot
2020-02-03 15:51:18.555309: [debug] Process finished in 34ms: /home/simon/.stack/programs/x86_64-linux/ghc-8.6.5/bin/ghc-pkg-8.6.5 --global --no-user-package-db dump --expand-pkgroot
2020-02-03 15:51:18.557048: [debug] Ignoring package haskeline due to wanting version 0.7.5.0 instead of 0.7.4.3
2020-02-03 15:51:18.557955: [debug] Ignoring package Cabal due to wanting version 2.4.1.0 instead of 2.4.0.1
2020-02-03 15:51:18.558599: [debug] Run process: /home/simon/.stack/programs/x86_64-linux/ghc-8.6.5/bin/ghc-pkg-8.6.5 --user --no-user-package-db --package-db /home/simon/.stack/snapshots/x86_64-linux/0343a892f720a3ab615a93dadf1fa84cde04daa105e6cc3e5bfcc8e7503626d6/8.6.5/pkgdb dump --expand-pkgroot
2020-02-03 15:51:18.614911: [debug] Process finished in 56ms: /home/simon/.stack/programs/x86_64-linux/ghc-8.6.5/bin/ghc-pkg-8.6.5 --user --no-user-package-db --package-db /home/simon/.stack/snapshots/x86_64-linux/0343a892f720a3ab615a93dadf1fa84cde04daa105e6cc3e5bfcc8e7503626d6/8.6.5/pkgdb dump --expand-pkgroot
2020-02-03 15:51:18.618513: [debug] Run process: /home/simon/.stack/programs/x86_64-linux/ghc-8.6.5/bin/ghc-pkg-8.6.5 --user --no-user-package-db --package-db /home/simon/src/prettyprinter/.stack-work/install/x86_64-linux/0343a892f720a3ab615a93dadf1fa84cde04daa105e6cc3e5bfcc8e7503626d6/8.6.5/pkgdb dump --expand-pkgroot
2020-02-03 15:51:18.639271: [debug] Process finished in 21ms: /home/simon/.stack/programs/x86_64-linux/ghc-8.6.5/bin/ghc-pkg-8.6.5 --user --no-user-package-db --package-db /home/simon/src/prettyprinter/.stack-work/install/x86_64-linux/0343a892f720a3ab615a93dadf1fa84cde04daa105e6cc3e5bfcc8e7503626d6/8.6.5/pkgdb dump --expand-pkgroot
2020-02-03 15:51:18.639718: [debug] Constructing the build plan
2020-02-03 15:51:18.641660: [debug] SELECT "id","directory","type","pkg_src","active","path_env_var","haddock" FROM "config_cache" WHERE "directory"=? AND "type"=?; [PersistText "/home/simon/src/prettyprinter/.stack-work/install/x86_64-linux/0343a892f720a3ab615a93dadf1fa84cde04daa105e6cc3e5bfcc8e7503626d6/8.6.5/",PersistText "lib:prettyprinter-1.6.0-KboYg0SVni9KoU0RW6n1M"]
2020-02-03 15:51:18.642431: [debug] SELECT "id", "config_cache_id", "index", "option" FROM "config_cache_dir_option" WHERE ("config_cache_id"=?) ORDER BY "index"; [PersistInt64 6]
2020-02-03 15:51:18.642629: [debug] SELECT "id", "config_cache_id", "index", "option" FROM "config_cache_no_dir_option" WHERE ("config_cache_id"=?) ORDER BY "index"; [PersistInt64 6]
2020-02-03 15:51:18.642834: [debug] SELECT "id", "config_cache_id", "ghc_pkg_id" FROM "config_cache_dep" WHERE ("config_cache_id"=?); [PersistInt64 6]
2020-02-03 15:51:18.642989: [debug] SELECT "id", "config_cache_id", "component" FROM "config_cache_component" WHERE ("config_cache_id"=?); [PersistInt64 6]
2020-02-03 15:51:18.643614: [debug] Start: getPackageFiles /home/simon/src/prettyprinter/prettyprinter/prettyprinter.cabal
2020-02-03 15:51:18.655952: [debug] Finished in 12ms: getPackageFiles /home/simon/src/prettyprinter/prettyprinter/prettyprinter.cabal
2020-02-03 15:51:18.657721: [debug] SELECT "id","directory","type","pkg_src","active","path_env_var","haddock" FROM "config_cache" WHERE "directory"=? AND "type"=?; [PersistText "/home/simon/src/prettyprinter/.stack-work/install/x86_64-linux/0343a892f720a3ab615a93dadf1fa84cde04daa105e6cc3e5bfcc8e7503626d6/8.6.5/",PersistText "lib:prettyprinter-ansi-terminal-1.1.1.2-Ir454Lit9VQItZJTrJmiAg"]
2020-02-03 15:51:18.657869: [debug] SELECT "id", "config_cache_id", "index", "option" FROM "config_cache_dir_option" WHERE ("config_cache_id"=?) ORDER BY "index"; [PersistInt64 142]
2020-02-03 15:51:18.658017: [debug] SELECT "id", "config_cache_id", "index", "option" FROM "config_cache_no_dir_option" WHERE ("config_cache_id"=?) ORDER BY "index"; [PersistInt64 142]
2020-02-03 15:51:18.658127: [debug] SELECT "id", "config_cache_id", "ghc_pkg_id" FROM "config_cache_dep" WHERE ("config_cache_id"=?); [PersistInt64 142]
2020-02-03 15:51:18.658207: [debug] SELECT "id", "config_cache_id", "component" FROM "config_cache_component" WHERE ("config_cache_id"=?); [PersistInt64 142]
2020-02-03 15:51:18.658543: [debug] Start: getPackageFiles /home/simon/src/prettyprinter/prettyprinter-ansi-terminal/prettyprinter-ansi-terminal.cabal
2020-02-03 15:51:18.660242: [debug] Finished in 2ms: getPackageFiles /home/simon/src/prettyprinter/prettyprinter-ansi-terminal/prettyprinter-ansi-terminal.cabal
2020-02-03 15:51:18.661256: [debug] SELECT "id","directory","type","pkg_src","active","path_env_var","haddock" FROM "config_cache" WHERE "directory"=? AND "type"=?; [PersistText "/home/simon/src/prettyprinter/.stack-work/install/x86_64-linux/0343a892f720a3ab615a93dadf1fa84cde04daa105e6cc3e5bfcc8e7503626d6/8.6.5/",PersistText "lib:prettyprinter-convert-ansi-wl-pprint-1.1-B8DNRiuB3wt6vuQ8VCWc67"]
2020-02-03 15:51:18.661523: [debug] SELECT "id", "config_cache_id", "index", "option" FROM "config_cache_dir_option" WHERE ("config_cache_id"=?) ORDER BY "index"; [PersistInt64 144]
2020-02-03 15:51:18.661665: [debug] SELECT "id", "config_cache_id", "index", "option" FROM "config_cache_no_dir_option" WHERE ("config_cache_id"=?) ORDER BY "index"; [PersistInt64 144]
2020-02-03 15:51:18.661778: [debug] SELECT "id", "config_cache_id", "ghc_pkg_id" FROM "config_cache_dep" WHERE ("config_cache_id"=?); [PersistInt64 144]
2020-02-03 15:51:18.661861: [debug] SELECT "id", "config_cache_id", "component" FROM "config_cache_component" WHERE ("config_cache_id"=?); [PersistInt64 144]
2020-02-03 15:51:18.662161: [debug] Start: getPackageFiles /home/simon/src/prettyprinter/prettyprinter-convert-ansi-wl-pprint/prettyprinter-convert-ansi-wl-pprint.cabal
2020-02-03 15:51:18.663000: [debug] Finished in 1ms: getPackageFiles /home/simon/src/prettyprinter/prettyprinter-convert-ansi-wl-pprint/prettyprinter-convert-ansi-wl-pprint.cabal
2020-02-03 15:51:18.663755: [debug] Checking if we are going to build multiple executables with the same name
2020-02-03 15:51:18.663812: [debug] Executing the build plan
2020-02-03 15:51:18.664669: [info] prettyprinter                       > test (suite: doctest)
2020-02-03 15:51:18.664715: [info] 
2020-02-03 15:51:18.665030: [info] prettyprinter-convert-ansi-wl-pprint> test (suite: doctest)
2020-02-03 15:51:18.665068: [info] 
2020-02-03 15:51:18.665173: [debug] Run process within /home/simon/src/prettyprinter/prettyprinter/: /home/simon/src/prettyprinter/prettyprinter/.stack-work/dist/x86_64-linux/Cabal-2.4.0.1/build/doctest/doctest
2020-02-03 15:51:18.665694: [debug] Run process within /home/simon/src/prettyprinter/prettyprinter-convert-ansi-wl-pprint/: /home/simon/src/prettyprinter/prettyprinter-convert-ansi-wl-pprint/.stack-work/dist/x86_64-linux/Cabal-2.4.0.1/build/doctest/doctest
2020-02-03 15:51:18.747818: [info] prettyprinter                       > Loaded package environment from /tmp/stack1839/test-ghc-env
2020-02-03 15:51:18.829275: [info] prettyprinter-convert-ansi-wl-pprint> Loaded package environment from /tmp/stack1839/test-ghc-env
2020-02-03 15:51:18.935629: [info] prettyprinter                       > Loaded package environment from /tmp/stack1839/test-ghc-env
2020-02-03 15:51:19.067489: [info] prettyprinter-convert-ansi-wl-pprint> Loaded package environment from /tmp/stack1839/test-ghc-env
2020-02-03 15:51:19.271818: [info] prettyprinter-convert-ansi-wl-pprint> Loaded package environment from /tmp/stack1839/test-ghc-env
2020-02-03 15:51:19.320910: [info] prettyprinter-convert-ansi-wl-pprint> Loaded package environment from /tmp/stack1839/test-ghc-env
2020-02-03 15:51:19.551563: [info] prettyprinter-convert-ansi-wl-pprint> Examples: 0  Tried: 0  Errors: 0  Failures: 0
2020-02-03 15:51:19.610981: [debug] Process finished in 945ms: /home/simon/src/prettyprinter/prettyprinter-convert-ansi-wl-pprint/.stack-work/dist/x86_64-linux/Cabal-2.4.0.1/build/doctest/doctest
2020-02-03 15:51:19.611057: [info] prettyprinter-convert-ansi-wl-pprint> Test suite doctest passed
2020-02-03 15:51:19.869187: [info] prettyprinter                       > Loaded package environment from /tmp/stack1839/test-ghc-env
2020-02-03 15:51:19.901947: [info] prettyprinter                       > Loaded package environment from /tmp/stack1839/test-ghc-env
2020-02-03 15:51:26.704818: [info] prettyprinter                       > src/Data/Text/Prettyprint/Doc/Internal.hs:1948: failure in expression `import Test.QuickCheck.Modifiers'
2020-02-03 15:51:26.704930: [info] prettyprinter                       > expected: 
2020-02-03 15:51:26.704987: [info] prettyprinter                       >  but got: 
2020-02-03 15:51:26.705051: [info] prettyprinter                       >           <no location info>: error:
2020-02-03 15:51:26.705107: [info] prettyprinter                       >               Could not load module ‘Test.QuickCheck.Modifiers’
2020-02-03 15:51:26.705155: [info] prettyprinter                       >               It is a member of the hidden package ‘QuickCheck-2.12.6.1’.
2020-02-03 15:51:26.705208: [info] prettyprinter                       >               You can run ‘:set -package QuickCheck’ to expose it.
2020-02-03 15:51:26.705261: [info] prettyprinter                       >               (Note: this unloads all the modules in the current scope.)
2020-02-03 15:51:26.705307: [info] prettyprinter                       > 
2020-02-03 15:51:26.708431: [info] prettyprinter                       > Examples: 249  Tried: 103  Errors: 0  Failures: 1
2020-02-03 15:51:26.800668: [debug] Process finished in 8135ms: /home/simon/src/prettyprinter/prettyprinter/.stack-work/dist/x86_64-linux/Cabal-2.4.0.1/build/doctest/doctest
2020-02-03 15:51:26.800749: [info] prettyprinter                       > Test suite doctest failed
2020-02-03 15:51:26.800835: [info] Completed 2 action(s).
2020-02-03 15:51:26.801380: [error] Test suite failure for package prettyprinter-1.6.0
    doctest:  exited with: ExitFailure 1
Logs printed to console

Note that the hidden QuickCheck package is a dependency of prettyprinter:doctest, but not prettyprinter-convert-ansi-wl-pprint:doctest.

Stack version

$ stack --version
Version 2.1.3, Git revision 636e3a759d51127df2b62f90772def126cdf6d1f (7735 commits) x86_64 hpack-0.31.2

Method of installation

Probably stack upgrade.
@qrilka
Copy link
Contributor

qrilka commented Feb 3, 2020

Can't say for sure but probably fixing #4745 should take care of this too

sjakobi added a commit to quchen/prettyprinter that referenced this issue Feb 8, 2020
@sjakobi
Prevent non-deterministic doctest failures
f331188 
This is a workaround for
commercialhaskell/stack#5159.
@pbrisbin pbrisbin mentioned this issue Jul 28, 2020
Merged
pbrisbin added a commit to freckle/bcp47 that referenced this issue Jul 28, 2020
@pbrisbin
Add unnecessary megaparsec dep to orphans tests
2205135 
There is a (long-standing) bug in Stack for multi-package projects where
their package environments are shared across tests. This means that if
bcp47-orphan's environment is used for bcp47's doctest suite, it can't
find megaparsec. If the bug reproduces or not depends on concurrency
settings and build order too.

See commercialhaskell/stack#5159

Adding the dependency in all environments ensures it always works. It's
seen as a weed in both places, so we ignore that via .weeder.yaml.
pbrisbin added a commit to freckle/bcp47 that referenced this issue Jul 28, 2020
@pbrisbin
Build packages separately
2eff735 
See commercialhaskell/stack#5159.
pbrisbin added a commit to freckle/bcp47 that referenced this issue Jul 28, 2020
@pbrisbin
Build packages separately
306b58a 
See commercialhaskell/stack#5159.
pbrisbin added a commit to freckle/bcp47 that referenced this issue Jul 28, 2020
@pbrisbin
Build packages separately
964071d 
See commercialhaskell/stack#5159.
@mpilgrem
Copy link
Member

I am closing given the passage of time, the closing of #4745, and because I could not reproduce the problem with the current version of Stack on Ubuntu (via WSL).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@qrilka @sjakobi @mpilgrem