Convert from shell to YAML-based build system

[natefoo]

Do others think this is a good idea? Should we do away with the shell script builds entirely?

[martenson]

I am not too familiar, can you please summarize what we gain with this conversion?

[natefoo]

More context is in #15. @erasche wrote an improvement where build instructions can be specified in YAML rather than writing shell scripts that do the build.

Initially we decided not to force this change since it made the whole thing more complex (and I give an example there where something I do fairly simply in a shell script would not be as easily done in YAML), but as Starforge has grown (and now has a generalized library and cli for building things), and the plan is to incorporate the YAML system into this (see #57), should we now push toward the YAML system for all packages?

Benefits are:

• Less copying
• Less hacky
• More structured
• Parseable version and other package metadata for automated updates

Drawbacks are as listed above.

[jxtx]

I think I like conda's mix of yaml for metadata and shell for build instructions. A list of shell commands in a yaml file is not much better than a shell script. At the same time, why should the shell script need to do everything (install build dependencies, download files). Seems like this could be separated.

[bgruening]

@natefoo I'm for yaml but I would like to redefine the yaml format to fit more into the conda format.
The is a lot of overlap between these projects and it would be nice to convert both into each other.

@jxtx is this also what you mean?

[jxtx]

Mostly. I don't think we should move everything into YAML. Really, why not just use the conda approach entirely.

[bgruening]

We probably don't need anything from the conda spec, but what is needed we should use. I really like it.

[bgruening]

@natefoo also https://github.com/bioconda/bioconda-recipes is always worth a look. They are also producing binaries with a CentoOS-5 Docker Image.

[natefoo]

I haven't used conda... for those of you that have, would it be conceivable to simply have Starforge use Conda to build packages in Docker? That way we end up with a lot less wheel reinvention.

[bgruening]

Conda as build system does not use or depend on Docker afaik, but biodocker is using a old Docker image to produce the binaries. I guess it should be fairly simple to extract the binary out of the conda package if we want to use it outside of conda as well.