Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cabal-install is twice as fast as Stack (on OS X with lots of packages: in stack.yaml) #2041

Closed
hamishmack opened this issue Apr 18, 2016 · 9 comments

Comments

@hamishmack
Copy link

I recently ported Leksah from Gtk2Hs to haskell-gi. When I started out I thought I would try using Stack, but it was very slow and I did not have time to investigate why (instead I went back to using cabal-install). The port is done now and I have time to help figure out what is going on.

To reproduce the issue I follow the instructions to build Leksah from GitHub with Cabal-install. I then ran stack build to also build it with Stack. I added an error to one of the source files to compare the time it takes cabal build and stack build to find the error. Stack is taking an extra 10-12 seconds on my 3 year old MacBook Pro with SSD.

I suspect it might be related to the large number of packages: in the stack.yaml file and perhaps the large number of source files in some of those packages (some have over a thousand hs and hsc files).

Cabal-install does not have to check all of those because Leksah tells it what package to build (there does not seem to be a stack command to build one package without checking the others).

~/haskell/leksah$ time cabal build
Building leksah-0.16.0.0...
Preprocessing library leksah-0.16.0.0...
[ 8 of 66] Compiling IDE.Core.State   ( src/IDE/Core/State.hs, dist/build/IDE/Core/State.o )

src/IDE/Core/State.hs:130:1-10:
    Parse error: naked expression at top level
    Perhaps you intended to use TemplateHaskell

real    0m10.334s
user    0m7.367s
sys 0m1.495s
~/haskell/leksah$ time stack build
There were multiple candidates for the Cabal entry "Main.hs" (/Users/hamish/haskell/leksah/vendor/haskellVCSGUI/vcsgui/src/Main.hs), picking /Users/hamish/haskell/leksah/vendor/haskellVCSGUI/vcsgui/src/exe/askpass/Main.hs
leksah-0.16.0.0: build

--  While building package leksah-0.16.0.0 using:
      /Users/hamish/.stack/setup-exe-cache/x86_64-osx/setup-Simple-Cabal-1.22.5.0-ghc-7.10.3 --builddir=.stack-work/dist/x86_64-osx/Cabal-1.22.5.0 build lib:leksah exe:bewleksah exe:leksah --ghc-options " -ddump-hi -ddump-to-file"
    Process exited with code: ExitFailure 1
    Logs have been written to: /Users/hamish/haskell/leksah/.stack-work/logs/leksah-0.16.0.0.log

    Preprocessing library leksah-0.16.0.0...
    [ 8 of 66] Compiling IDE.Core.State   ( src/IDE/Core/State.hs, .stack-work/dist/x86_64-osx/Cabal-1.22.5.0/build/IDE/Core/State.o )

    src/IDE/Core/State.hs:130:1-10:
        Parse error: naked expression at top level
        Perhaps you intended to use TemplateHaskell

real    0m26.883s
user    0m18.065s
sys 0m4.673s

Here it is with verbose logging

~/haskell/leksah$ time stack build -v
Version 1.0.4.1 x86_64
2016-04-18 18:41:13.056723: [debug] Checking for project config at: /Users/hamish/haskell/leksah/stack.yaml @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Config src/Stack/Config.hs:761:9)
2016-04-18 18:41:13.065820: [debug] Loading project config file stack.yaml @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Config src/Stack/Config.hs:779:13)
2016-04-18 18:41:13.067967: [debug] Trying to decode /Users/hamish/.stack/build-plan-cache/x86_64-osx/lts-5.4.cache @(stack_A0MVXjHMqxMB3zqs1HUvul:Data.Binary.VersionTagged src/Data/Binary/VersionTagged.hs:55:5)
2016-04-18 18:41:13.089190: [debug] Success decoding /Users/hamish/.stack/build-plan-cache/x86_64-osx/lts-5.4.cache @(stack_A0MVXjHMqxMB3zqs1HUvul:Data.Binary.VersionTagged src/Data/Binary/VersionTagged.hs:64:13)
2016-04-18 18:41:13.089384: [debug] Trying to decode /Users/hamish/.stack/indices/Hackage/00-index.cache @(stack_A0MVXjHMqxMB3zqs1HUvul:Data.Binary.VersionTagged src/Data/Binary/VersionTagged.hs:55:5)
2016-04-18 18:41:13.471505: [debug] Success decoding /Users/hamish/.stack/indices/Hackage/00-index.cache @(stack_A0MVXjHMqxMB3zqs1HUvul:Data.Binary.VersionTagged src/Data/Binary/VersionTagged.hs:64:13)
2016-04-18 18:41:13.485568: [debug] Run process: ghc --info @(stack_A0MVXjHMqxMB3zqs1HUvul:System.Process.Read src/System/Process/Read.hs:269:3)
2016-04-18 18:41:13.606656: [debug] Run process: ghc --numeric-version @(stack_A0MVXjHMqxMB3zqs1HUvul:System.Process.Read src/System/Process/Read.hs:269:3)
2016-04-18 18:41:13.695964: [debug] Run process: ghc-pkg --no-user-package-db field --simple-output Cabal version @(stack_A0MVXjHMqxMB3zqs1HUvul:System.Process.Read src/System/Process/Read.hs:269:3)
2016-04-18 18:41:13.762865: [debug] Run process: ghc-pkg --no-user-package-db list --global @(stack_A0MVXjHMqxMB3zqs1HUvul:System.Process.Read src/System/Process/Read.hs:269:3)
2016-04-18 18:41:14.267263: [debug] Checking resolver: lts-5.4 @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Source src/Stack/Build/Source.hs:164:17)
2016-04-18 18:41:14.267471: [debug] Trying to decode /Users/hamish/.stack/build-plan-cache/x86_64-osx/lts-5.4.cache @(stack_A0MVXjHMqxMB3zqs1HUvul:Data.Binary.VersionTagged src/Data/Binary/VersionTagged.hs:55:5)
2016-04-18 18:41:14.277482: [debug] Success decoding /Users/hamish/.stack/build-plan-cache/x86_64-osx/lts-5.4.cache @(stack_A0MVXjHMqxMB3zqs1HUvul:Data.Binary.VersionTagged src/Data/Binary/VersionTagged.hs:64:13)
2016-04-18 18:41:26.099900: [warn] There were multiple candidates for the Cabal entry "Main.hs" (/Users/hamish/haskell/leksah/vendor/haskellVCSGUI/vcsgui/src/Main.hs), picking /Users/hamish/haskell/leksah/vendor/haskellVCSGUI/vcsgui/src/exe/askpass/Main.hs @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Package src/Stack/Package.hs:1014:5)
2016-04-18 18:41:26.230448: [debug] Run process: ghc-pkg --global --no-user-package-db dump --expand-pkgroot @(stack_A0MVXjHMqxMB3zqs1HUvul:System.Process.Read src/System/Process/Read.hs:269:3)
2016-04-18 18:41:26.407030: [debug] Ignoring package haskeline due to wanting version 0.7.2.2 instead of 0.7.2.1 @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.407581: [debug] Ignoring package terminfo due to wanting version 0.4.0.2 instead of 0.4.0.1 @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.407757: [debug] Ignoring package Cabal due to wanting version 1.22.7.0 instead of 1.22.5.0 @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.408428: [debug] Run process: ghc-pkg --user --no-user-package-db --package-db /Users/hamish/.stack/snapshots/x86_64-osx/lts-5.4/7.10.3/pkgdb dump --expand-pkgroot @(stack_A0MVXjHMqxMB3zqs1HUvul:System.Process.Read src/System/Process/Read.hs:269:3)
2016-04-18 18:41:26.518119: [debug] Ignoring package lens, from (InstalledTo Snap,"/Users/hamish/.stack/snapshots/x86_64-osx/lts-5.4/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.518421: [debug] Ignoring package yaml, from (InstalledTo Snap,"/Users/hamish/.stack/snapshots/x86_64-osx/lts-5.4/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.518521: [debug] Ignoring package kan-extensions, from (InstalledTo Snap,"/Users/hamish/.stack/snapshots/x86_64-osx/lts-5.4/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.518600: [debug] Ignoring package haskell-src-meta, from (InstalledTo Snap,"/Users/hamish/.stack/snapshots/x86_64-osx/lts-5.4/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.518913: [debug] Ignoring package adjunctions, from (InstalledTo Snap,"/Users/hamish/.stack/snapshots/x86_64-osx/lts-5.4/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.519033: [debug] Ignoring package haskell-src-exts, from (InstalledTo Snap,"/Users/hamish/.stack/snapshots/x86_64-osx/lts-5.4/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.519116: [debug] Ignoring package aeson, from (InstalledTo Snap,"/Users/hamish/.stack/snapshots/x86_64-osx/lts-5.4/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.519322: [debug] Ignoring package free, from (InstalledTo Snap,"/Users/hamish/.stack/snapshots/x86_64-osx/lts-5.4/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.519403: [debug] Ignoring package semigroupoids, from (InstalledTo Snap,"/Users/hamish/.stack/snapshots/x86_64-osx/lts-5.4/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.519486: [debug] Ignoring package profunctors, from (InstalledTo Snap,"/Users/hamish/.stack/snapshots/x86_64-osx/lts-5.4/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.519566: [debug] Ignoring package bifunctors, from (InstalledTo Snap,"/Users/hamish/.stack/snapshots/x86_64-osx/lts-5.4/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.519644: [debug] Ignoring package comonad, from (InstalledTo Snap,"/Users/hamish/.stack/snapshots/x86_64-osx/lts-5.4/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.519722: [debug] Ignoring package attoparsec, from (InstalledTo Snap,"/Users/hamish/.stack/snapshots/x86_64-osx/lts-5.4/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.519873: [debug] Ignoring package contravariant, from (InstalledTo Snap,"/Users/hamish/.stack/snapshots/x86_64-osx/lts-5.4/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.519953: [debug] Ignoring package void, from (InstalledTo Snap,"/Users/hamish/.stack/snapshots/x86_64-osx/lts-5.4/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.520031: [debug] Ignoring package shelly, from (InstalledTo Snap,"/Users/hamish/.stack/snapshots/x86_64-osx/lts-5.4/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.520109: [debug] Ignoring package semigroups, from (InstalledTo Snap,"/Users/hamish/.stack/snapshots/x86_64-osx/lts-5.4/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.520185: [debug] Ignoring package HTTP, from (InstalledTo Snap,"/Users/hamish/.stack/snapshots/x86_64-osx/lts-5.4/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.520263: [debug] Ignoring package cpphs, from (InstalledTo Snap,"/Users/hamish/.stack/snapshots/x86_64-osx/lts-5.4/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.520341: [debug] Ignoring package unordered-containers, from (InstalledTo Snap,"/Users/hamish/.stack/snapshots/x86_64-osx/lts-5.4/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.520499: [debug] Ignoring package network-uri, from (InstalledTo Snap,"/Users/hamish/.stack/snapshots/x86_64-osx/lts-5.4/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.520581: [debug] Ignoring package lifted-async, from (InstalledTo Snap,"/Users/hamish/.stack/snapshots/x86_64-osx/lts-5.4/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.520660: [debug] Ignoring package system-fileio, from (InstalledTo Snap,"/Users/hamish/.stack/snapshots/x86_64-osx/lts-5.4/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.520736: [debug] Ignoring package polyparse, from (InstalledTo Snap,"/Users/hamish/.stack/snapshots/x86_64-osx/lts-5.4/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.520822: [debug] Ignoring package constraints, from (InstalledTo Snap,"/Users/hamish/.stack/snapshots/x86_64-osx/lts-5.4/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.520898: [debug] Ignoring package system-filepath, from (InstalledTo Snap,"/Users/hamish/.stack/snapshots/x86_64-osx/lts-5.4/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.521038: [debug] Ignoring package scientific, from (InstalledTo Snap,"/Users/hamish/.stack/snapshots/x86_64-osx/lts-5.4/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.521118: [debug] Ignoring package parsec, from (InstalledTo Snap,"/Users/hamish/.stack/snapshots/x86_64-osx/lts-5.4/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.521195: [debug] Ignoring package wl-pprint-text, from (InstalledTo Snap,"/Users/hamish/.stack/snapshots/x86_64-osx/lts-5.4/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.521383: [debug] Ignoring package text-binary, from (InstalledTo Snap,"/Users/hamish/.stack/snapshots/x86_64-osx/lts-5.4/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.521526: [debug] Ignoring package hashable, from (InstalledTo Snap,"/Users/hamish/.stack/snapshots/x86_64-osx/lts-5.4/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.521618: [debug] Ignoring package text, from (InstalledTo Snap,"/Users/hamish/.stack/snapshots/x86_64-osx/lts-5.4/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.522424: [debug] Run process: ghc-pkg --user --no-user-package-db --package-db /Users/hamish/haskell/leksah/.stack-work/install/x86_64-osx/lts-5.4/7.10.3/pkgdb dump --expand-pkgroot @(stack_A0MVXjHMqxMB3zqs1HUvul:System.Process.Read src/System/Process/Read.hs:269:3)
2016-04-18 18:41:26.658434: [debug] Ignoring package gtk3, from (InstalledTo Local,"/Users/hamish/haskell/leksah/.stack-work/install/x86_64-osx/lts-5.4/7.10.3/pkgdb/"), due to wanting version 0.14.2 instead of 0.15.0 @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-04-18 18:41:26.659582: [debug] Trying to decode /Users/hamish/.stack/indices/Hackage/00-index.cache @(stack_A0MVXjHMqxMB3zqs1HUvul:Data.Binary.VersionTagged src/Data/Binary/VersionTagged.hs:55:5)
2016-04-18 18:41:27.014936: [debug] Success decoding /Users/hamish/.stack/indices/Hackage/00-index.cache @(stack_A0MVXjHMqxMB3zqs1HUvul:Data.Binary.VersionTagged src/Data/Binary/VersionTagged.hs:64:13)
2016-04-18 18:41:27.275996: [debug] Run process: ghc-pkg --no-user-package-db list --global @(stack_A0MVXjHMqxMB3zqs1HUvul:System.Process.Read src/System/Process/Read.hs:269:3)
2016-04-18 18:41:27.336329: [info] leksah-0.16.0.0: build @(stack_A0MVXjHMqxMB3zqs1HUvul:Stack.Build.Execute src/Stack/Build/Execute.hs:703:23)
2016-04-18 18:41:27.336509: [debug] Run process: /Users/hamish/.stack/setup-exe-cache/x86_64-osx/setup-Simple-Cabal-1.22.5.0-ghc-7.10.3 --builddir=.stack-work/dist/x86_64-osx/Cabal-1.22.5.0 build lib:leksah exe:bewleksah exe:leksah --ghc-options " -ddump-hi -ddump-to-file" @(stack_A0MVXjHMqxMB3zqs1HUvul:System.Process.Read src/System/Process/Read.hs:287:3)

--  While building package leksah-0.16.0.0 using:
      /Users/hamish/.stack/setup-exe-cache/x86_64-osx/setup-Simple-Cabal-1.22.5.0-ghc-7.10.3 --builddir=.stack-work/dist/x86_64-osx/Cabal-1.22.5.0 build lib:leksah exe:bewleksah exe:leksah --ghc-options " -ddump-hi -ddump-to-file"
    Process exited with code: ExitFailure 1
    Logs have been written to: /Users/hamish/haskell/leksah/.stack-work/logs/leksah-0.16.0.0.log

    Preprocessing library leksah-0.16.0.0...
    [ 8 of 66] Compiling IDE.Core.State   ( src/IDE/Core/State.hs, .stack-work/dist/x86_64-osx/Cabal-1.22.5.0/build/IDE/Core/State.o )

    src/IDE/Core/State.hs:130:1-10:
        Parse error: naked expression at top level
        Perhaps you intended to use TemplateHaskell

real    0m25.982s
user    0m17.062s
sys 0m4.598s
@mgsloan
Copy link
Contributor

mgsloan commented Apr 18, 2016

Interesting, thanks for the test case. You're probably on the right track that it's related to dirtiness checking thousands of files. It should be possible to do dirtiness checking only for packages that are actually demanded by stack build. Actually, I thought that was already how it was done. Are you sure it doesn't work that way?

cabal-install doesn't need to do this work, because you need to build each local package independently. So you're already telling it "ignore changes in those other packages".

I guess there could be some sort of --ignore-dirty flag that lets you build a specific package and not check other packages for dirtiness (even deps). Yuck!

@mgsloan mgsloan added this to the P2: Should milestone Apr 18, 2016
@hamishmack
Copy link
Author

hamishmack commented Apr 19, 2016

How about including a dirty mark for each package and having:

  • stack check [PACKAGE] check for modification and mark as dirty the package(s) that have changed on disk.
  • stack touch PACKAGE manually mark a package as dirty without even looking to see if it really is.
  • stack build --marked would do the build for the packages marked.

So stack check && stack build --marked would be the same as stack build.

Leksah though would call stack touch PACKAGE when user has modified a file in PACKAGE and then call stack build --marked. We can interrupt stack build --marked with SIGINT, but avoid interrupting stack touch PACKAGE.

As a bonus both stack check and stack build would not have to check for modifications in packages already marked as dirty.

@mgsloan
Copy link
Contributor

mgsloan commented Apr 19, 2016

So far we have avoided any sort of configuration state - it's all in the stack.yaml or the immediate CLI. This makes things more reproducible and reliable. This would be a lot of added complexity to handle a rare case.

How about we have stack build --only-targets-dirty pkg1 pkg2? This will force dirtiness for pkg1 and pkg2, and skip the check for other local packages. I think this is more consistent with stack's usual CLI style.

@hamishmack
Copy link
Author

hamishmack commented Apr 20, 2016

Having stack build --only-targets-dirty pkg1 pkg2 would be fine.

It would be nice though to be able to persist the dirty state somehow (so that restarting Leksah does not forget what packages were dirty).

Is there a good way to get a timestamp for the last successful build of a package?

I figure we might be able to do touch PACKAGE.cabal instead of stack touch PACKAGE (or failing that we can touch some dummy file).

Then we could find all the packages Leksah needs to pass to stack build --only-targets-dirty by comparing the timestamp of the PACKAGE.cabal with the timestamp of the last successful build of PACKAGE.

@mgsloan
Copy link
Contributor

mgsloan commented Apr 20, 2016

If you want leksah's use of stack to be stateful, you're going to need to store state.

I don't think there's a good way to get a timestamp for the last successful build - could go digging around for timestamps in dist, though.

@hamishmack
Copy link
Author

If you want leksah's use of stack to be stateful, you're going to need to store state.

Yep, that is fine stack build --only-targets-dirty pkg1 pkg2 will be fine. We can do the rest in Leksah.

I don't think there's a good way to get a timestamp for the last successful build - could go digging around for timestamps in dist, though.

I think we can get it from .stack-work/logs. If the log file ends in Registering PACKAGE-1.0 then it was successful and timestamp of the log file is the timestamp we need to compare. If the log file is missing or does not end with Registering PACKAGE-1.0 then the package is dirty.

@mgsloan
Copy link
Contributor

mgsloan commented Apr 20, 2016

Note that currently logs are not outputted in the case that the build output goes to the console - see the recent discussion #2045

@hamishmack
Copy link
Author

This still seems to be a significant issue. I have a faster machine now, but stack is still slower than cabal-install (8.348s vs 2.503s).

~/haskell/leksah$ time cabal new-build
In order, the following will be built (use -v for more details):
 - leksah-0.16.2.2 (lib) (file src/IDE/Core/State.hs changed)
 - leksah-0.16.2.2 (exe:leksah) (dependency rebuilt)
Preprocessing library for leksah-0.16.2.2..
Building library for leksah-0.16.2.2..
[10 of 67] Compiling IDE.Core.State   ( src/IDE/Core/State.hs, /Users/hamish/haskell/leksah/dist-newstyle/build/x86_64-osx/ghc-8.2.1/leksah-0.16.2.2/build/IDE/Core/State.o )

src/IDE/Core/State.hs:136:1-21: error:
    Parse error: module header, import declaration
    or top-level declaration expected.
    |
136 | cdscdsv vds d vds v d
    | ^^^^^^^^^^^^^^^^^^^^^
cabal: Failed to build leksah-0.16.2.2 (which is required by exe:leksah from
leksah-0.16.2.2).

real	0m2.503s
user	0m1.870s
sys	0m0.440s
~/haskell/leksah$ time PATH=/Users/hamish/ghc-8.0.2/bin:$PATH stack build --stack-yaml=stack.osx.yaml leksah:exe:leksah
There were multiple candidates for the Cabal entry "Main.hs" (/Users/hamish/haskell/leksah/vendor/haskellVCSGUI/vcsgui/src/Main.hs), picking /Users/hamish/haskell/leksah/vendor/haskellVCSGUI/vcsgui/src/exe/askpass/Main.hs
leksah-0.16.2.2: build (lib + exe)
Preprocessing library leksah-0.16.2.2...
[10 of 67] Compiling IDE.Core.State   ( src/IDE/Core/State.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/IDE/Core/State.o )

/Users/hamish/haskell/leksah/src/IDE/Core/State.hs:136:1-21: error:
    Parse error: naked expression at top level
    Perhaps you intended to use TemplateHaskell

--  While building package leksah-0.16.2.2 using:
      /Users/hamish/.stack/setup-exe-cache/x86_64-osx/Cabal-simple_mPHDZzAJ_1.24.2.0_ghc-8.0.2 --builddir=.stack-work/dist/x86_64-osx/Cabal-1.24.2.0 build lib:leksah exe:leksah --ghc-options " -ddump-hi -ddump-to-file"
    Process exited with code: ExitFailure 1

real	0m8.348s
user	0m7.245s
sys	0m1.006s

@mgsloan
Copy link
Contributor

mgsloan commented Sep 17, 2017

Your stack.yaml is using a bunch of local packages which . Since these packages are local and may be modified, it is collecting a list of all files. Some of these packages are automatically generated and have many hundreds of modules. Unfortunately, this files collection code is currently inefficient - this is discussed and tracked in the latter part of #1235 , so closing this as a dupe of that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants