comment out failing tests, fixes #217

[gelisam]

I would much rather fix those tests, but this is the more expedient way to proceed for now.

[gelisam]

The failure was

tests/System/Console/Hawk/PreludeTests.hs:43: failure in expression `test ["-d:", "-m"] "head" "passwd"'
expected: root
 but got: warning: 
            Loaded package environment from /tmp/stack20081/test-ghc-env
            Loaded package environment from /tmp/stack20081/test-ghc-env
          root

These warnings seem benign, but I don't know how to tweak the test to ignore them. So I'll comment it out for now and bring the test back in #220.

[gelisam]

Still failing. This time the failure is

tests/System/Console/Hawk/PreludeTests.hs:60: failure in expression `testEval [] "[1..3]"'
expected: 1
          2
          3
 but got: warning: 
            Loaded package environment from /tmp/stack20140/test-ghc-env
            Loaded package environment from /tmp/stack20140/test-ghc-env
          1
          2
          3

I'm afraid this means every single test is going to fail this way one after the other, so commenting out the tests which fail would imply disabling testing altogether. Is there a less drastic solution?

[gelisam]

Progress! Two out of three builds used to fail, now there's one one which fails. It fails while building happy, but the error message doesn't make much sense. Does happy need happy in order to build successfully?

happy                > Configuring happy-1.19.5...
happy                > build
happy                > Building happy-1.19.5...
happy                > Preprocessing executable 'happy' for happy-1.19.5...
happy                > setup: The program 'happy' is required but it could not be found

[gelisam]

...which is pretty strange, because it is only the cabal build, which succeeds, which is installing happy, the stack build which fails is building hawk directly. I guess we must have happy as an indirect dependency. Maybe I need to tweak the .travis file to install happy separately, like I do in the cabal build?

[gelisam]

I can reproduce the problem by clearing part of my ~/.stack and typing stack --resolver="lts-8.0" install happy. I am thus guessing that this is a stack regression.

[gelisam]

Indeed, that bug was recently reported: commercialhaskell/stack#5253

[gelisam]

The reason I am building with lts-8.0 on CI is to ensure that my lower bounds aren't too low.

I don't care if my bounds are a bit too high; but I really don't want cabal to think it found a good build plan which turns out to actually be a bad plan much later in the build because of some lies I told it, that's terrible UX. If cabal thinks it cannot find a build plan at all, that's still frustrating, but at least there is a clear explanation: two dependencies are incompatible.

To check that my lower bounds are correct, I would ideally use kabal, but since that has never been released nor incorporated into cabal as an option, I decided this approach is too difficult. Another approach would be to use a cabal freeze file which sets all my dependencies to the lowest version I support; this way, if the build which uses that freeze file fails, I'll know it's because my bounds are too low.

The problem with that approach is that I then have to construct a new freeze file with slightly higher dependencies, and since packages sometimes need to be upgraded in unison, it's not as easy as bumping the bounds of one package at a time.

This, finally, is why I am building with lts-8.0. It's the oldest lts on which hawk builds successfully, and so I put my lower bounds to the versions included in lts-8.0, even though that means some of those bounds are probably a bit to high. If that build breaks, I know I've changed the code in a way which now uses a feature which is only available in more recent versions of my dependencies, and so I need to bump my lower bounds. With this approach, doing so is easy: I only have one number to bump, the lts, and once I get a build which passes, I update all my lower bounds to use the versions of that lts. Easy!

Now, the reason the lts-8.0 build broke this time is not because I have changed the code in that way, but because stack is buggy. But since I don't care if my lower bounds are a bit too high, I think I should be able to work around the problem by unnecessarily bumping to lts-9.0.

[gelisam]

stack --resolver="lts-9.0" install happy fails too. lts-10.0?

[gelisam]

It builds. But I now have another idea! According to this comment, the reason happy fails to build is because it wants happy to already be in the PATH. So another way to fix the problem should be to install (a newer version of) happy separately, just like I do when building with cabal.