Integration test for Koans archetype that runs koans

[DavidWhitlock]

Now that I've figured out how to write an integration test for an archetype with the "grader" profile enabled (as seen in issue #197). I should write an integration test for the koans archetype that executes the exec:java goal with the grader profile enabled and then verify that the expected output was written to the integration tests build.log.

[DavidWhitlock]

I wrote the test, but it's not passing because the JVM returns an exit code of 255 when there is any failing koan. When the "interactive = false" feature was added to the koans (matyb/java-koans#57) @matyb mentioned that the koans JVM would return a non-zero code when one of them failed. That sounds really reasonable, but I don't know if I'll be able to get the integration test to pass.

It looks the IntegrationTestMojo for archetypes always blows up when a non-zero exit code occurs: https://github.com/lalyos/maven-archetype-plugin/blob/master/src/main/java/org/apache/maven/archetype/mojos/IntegrationTestMojo.java#L500

Maybe I should modified the koans infrastructure to exit with a non-zero code under specific circumstances. Hmm.

[matyb]

nice to see there's still use cases i hadn't considered :)

not really sure how to avoid that without weirding up your build or modifying the koans lib itself. just off the cuff, had these ideas:

1. replace this magic number (255) with a configurable number or a field you reflectively setup (255 ->0) and teardown(0 -> 255) before executing that test instead
2. if this step only runs outside typical build & verify, between fetching the updated source and executing that step - you could rm -r the koan's source folder so it finds no koans to fail

[DavidWhitlock]

Hey, @matyb. I was leaning towards the first option. Thank you for the breadcrumbs. I'll give it a try on my fork and, if it works for me, I'll put up a pull request on the upstream.