Ruby tests failing for Python PR

[Zac-HD]

https://travis-ci.org/HypothesisWorks/hypothesis/jobs/396075133 failed while building #1282. Perhaps we should force-pass Ruby tests for PRs that don't touch hypothesis-ruby/ until it leaves alpha?

[DRMacIver]

Perhaps we should force-pass Ruby tests for PRs that don't touch hypothesis-ruby/ until it leaves alpha?

Given how much flakiness we're seeing with the Python build, I'm not sure the Ruby build is likely to be more of a problem here than the Python build.

The real question is why these tests are being run here at all. The if_changed logic is supposed to stop that happening.

[alexwlchan]

FWIW, to check for file changes in the Wellcome repo, I use

$ git diff --name-only [...]

and run checks against the list of files I get back. (Yes you shouldn't assume filenames don't contain newlines and splitting a list of filenames on newlines is bad, but in a repo with moderately sensible devs this is good enough.)

That allows our Travis scripts to dump an explanation of why they did or didn't run tests. I find that helpful when I'm fine-tuning the logic, maybe something similar would be helpful here?

## Reasons to run tests ##

Changes to Scala common libs affect Scala apps:
 - sbt_common/elasticsearch/src/main/scala/uk/ac/wellcome/elasticsearch/WorksIndex.scala
 - sbt_common/internal_model/src/main/scala/uk/ac/wellcome/models/work/internal/IdentityState.scala
 - sbt_common/internal_model/src/main/scala/uk/ac/wellcome/models/work/internal/Item.scala
 - sbt_common/internal_model/src/main/scala/uk/ac/wellcome/models/work/internal/MultipleSourceIdentifiers.scala
 - sbt_common/internal_model/src/main/scala/uk/ac/wellcome/models/work/internal/Work.scala
 - sbt_common/internal_model/src/test/scala/uk/ac/wellcome/models/work/test/util/WorksUtil.scala

Path is an exclusive dependency of this build task:
 - catalogue_pipeline/transformer/src/main/scala/uk/ac/wellcome/platform/transformer/transformers/CalmTransformableTransformer.scala
 - catalogue_pipeline/transformer/src/main/scala/uk/ac/wellcome/platform/transformer/transformers/MiroTransformableTransformer.scala
 - catalogue_pipeline/transformer/src/main/scala/uk/ac/wellcome/platform/transformer/transformers/SierraTransformableTransformer.scala
 - catalogue_pipeline/transformer/src/main/scala/uk/ac/wellcome/platform/transformer/transformers/miro/MiroIdentifiers.scala
 - catalogue_pipeline/transformer/src/main/scala/uk/ac/wellcome/platform/transformer/transformers/sierra/SierraConcepts.scala
 - catalogue_pipeline/transformer/src/main/scala/uk/ac/wellcome/platform/transformer/transformers/sierra/SierraContributors.scala
 - catalogue_pipeline/transformer/src/main/scala/uk/ac/wellcome/platform/transformer/transformers/sierra/SierraIdentifiers.scala
 - catalogue_pipeline/transformer/src/main/scala/uk/ac/wellcome/platform/transformer/transformers/sierra/SierraItems.scala
 - catalogue_pipeline/transformer/src/test/scala/uk/ac/wellcome/platform/transformer/receive/NotificationMessageReceiverTest.scala
 - catalogue_pipeline/transformer/src/test/scala/uk/ac/wellcome/platform/transformer/transformers/SierraTransformableTransformerTest.scala
 - catalogue_pipeline/transformer/src/test/scala/uk/ac/wellcome/platform/transformer/transformers/miro/MiroIdentifiersTest.scala
 - catalogue_pipeline/transformer/src/test/scala/uk/ac/wellcome/platform/transformer/transformers/sierra/SierraConceptsTest.scala
 - catalogue_pipeline/transformer/src/test/scala/uk/ac/wellcome/platform/transformer/transformers/sierra/SierraContributorsTest.scala
 - catalogue_pipeline/transformer/src/test/scala/uk/ac/wellcome/platform/transformer/transformers/sierra/SierraIdentifiersTest.scala


## Reasons not to run tests ##

Path is an exclusive dependency of a different build task (id_minter):
 - catalogue_pipeline/id_minter/src/main/scala/uk/ac/wellcome/platform/idminter/steps/IdEmbedder.scala
 - catalogue_pipeline/id_minter/src/test/scala/uk/ac/wellcome/platform/idminter/IdMinterFeatureTest.scala
 - catalogue_pipeline/id_minter/src/test/scala/uk/ac/wellcome/platform/idminter/steps/IdEmbedderTests.scala

Path is an exclusive dependency of a different build task (matcher):
 - catalogue_pipeline/matcher/src/test/scala/uk/ac/wellcome/platform/matcher/MatcherFeatureTest.scala
 - catalogue_pipeline/matcher/src/test/scala/uk/ac/wellcome/platform/matcher/fixtures/MatcherFixtures.scala
 - catalogue_pipeline/matcher/src/test/scala/uk/ac/wellcome/platform/matcher/matcher/WorkMatcherTest.scala
 - catalogue_pipeline/matcher/src/test/scala/uk/ac/wellcome/platform/matcher/messages/MatcherMessageReceiverTest.scala

Path is an exclusive dependency of a different build task (snapshot_generator):
 - data_api/snapshot_generator/src/test/scala/uk/ac/wellcome/platform/snapshot_generator/SnapshotGeneratorFeatureTest.scala

Changes to the display library don't affect this task:
 - sbt_common/display/src/main/scala/uk/ac/wellcome/display/models/v1/DisplayAgentV1.scala
 - sbt_common/display/src/main/scala/uk/ac/wellcome/display/models/v1/DisplayConceptV1.scala
 - sbt_common/display/src/main/scala/uk/ac/wellcome/display/models/v2/DisplayAbstractAgentV2.scala
 - sbt_common/display/src/main/scala/uk/ac/wellcome/display/models/v2/DisplayAbstractConcept.scala
 - sbt_common/display/src/test/scala/uk/ac/wellcome/display/models/DisplaySerialisationTestBase.scala
 - sbt_common/display/src/test/scala/uk/ac/wellcome/display/models/v1/DisplayAgentV1Test.scala
 - sbt_common/display/src/test/scala/uk/ac/wellcome/display/models/v1/DisplayItemV1Test.scala
 - sbt_common/display/src/test/scala/uk/ac/wellcome/display/models/v1/DisplayWorkV1SerialisationTest.scala
 - sbt_common/display/src/test/scala/uk/ac/wellcome/display/models/v1/DisplayWorkV1Test.scala
 - sbt_common/display/src/test/scala/uk/ac/wellcome/display/models/v2/DisplayAbstractConceptSerialisationTest.scala
 - sbt_common/display/src/test/scala/uk/ac/wellcome/display/models/v2/DisplayConceptTest.scala
 - sbt_common/display/src/test/scala/uk/ac/wellcome/display/models/v2/DisplayGenreV2SerialisationTest.scala
 - sbt_common/display/src/test/scala/uk/ac/wellcome/display/models/v2/DisplayItemV2Test.scala
 - sbt_common/display/src/test/scala/uk/ac/wellcome/display/models/v2/DisplaySubjectV2SerialisationTest.scala
 - sbt_common/display/src/test/scala/uk/ac/wellcome/display/models/v2/DisplayWorkV2SerialisationTest.scala
 - sbt_common/display/src/test/scala/uk/ac/wellcome/display/models/v2/DisplayWorkV2Test.scala

Path is an exclusive dependency of a different build task (api):
 - catalogue_api/api/src/test/scala/uk/ac/wellcome/platform/api/works/v1/ApiV1WorksTest.scala
 - catalogue_api/api/src/test/scala/uk/ac/wellcome/platform/api/works/v2/ApiV2WorksTest.scala

Path is an exclusive dependency of a different build task (recorder):
 - catalogue_pipeline/recorder/src/test/scala/uk/ac/wellcome/platform/recorder/RecorderFeatureTest.scala
 - catalogue_pipeline/recorder/src/test/scala/uk/ac/wellcome/platform/recorder/services/RecorderWorkerServiceTest.scala

Path is an exclusive dependency of a different build task (ingestor):
 - catalogue_pipeline/ingestor/src/test/scala/uk/ac/wellcome/platform/ingestor/IngestorFeatureTest.scala
 - catalogue_pipeline/ingestor/src/test/scala/uk/ac/wellcome/platform/ingestor/services/CustomElasticSearchMapping.scala

*** We're going to run tests

[Zac-HD]

Given how much flakiness we're seeing with the Python build, I'm not sure the Ruby build is likely to be more of a problem here than the Python build.

Blame that on the sudden increase in test power thanks to #1295 - it turns out that a handful of tests reliably passed, but only because we almost always explored the same states. This should calm down soon, once we improve the remaining test issues (e.g. #1348, and discussed in #1330).

More specifically, I'm happy to call this issue closed by #1354.