Create a workflow test for Optimus using the dsde-pipelines workflow test infrastructure

[kbergin]

To better allow for iterative testing of the Optimus pipeline, we want a workflow test that only runs optimus (vs running as an integration test) and validates the workflow output. Using the dsde-pipelines test framework provides a way to do this.

Why do it? To facilitate iterative development of Optimus
Where to start? In dsde-pipelines, subclass CromwellWorkflowTester and add a new environment to TestCromwellEnvironment
When is it done? A running workflow test that can be run using sbt run

Upstream dependencies: requires inputs, options to run Optimus
Downstream dependencies: Verify test output with scientific owner, store as truth data, create an Optimus validation WDL, add to nightly workflow test in jenkins. These can all be split into other tasks as necessary.

To solve the problem of cross-repo dependencies, the Optimus test can be written to assume that the skylab repo is checked out in a known relative location to the dsde-pipelines repo. If this repo isn't present the test will fail with an error.

┆Issue is synchronized with this Jira Dev Task

[kbergin]

Of note, we’ve made Optimus testing data and it is being used by Hornet in their testing for the DCP. May be helpful re: upstream dependencies and storing truth data. Nick, Kishori, Rex, Saman would be good contacts for Q’s

[kbergin]

Postponed because in the short term we are going to use the existing CI infrastructure in Skylab for the HCA pipelines.