-
Notifications
You must be signed in to change notification settings - Fork 37
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update to Omnia Linux anvil #761
Changes from 23 commits
831fd41
38c83df
9e3a83e
3956f2a
7d80095
81ff08d
1141426
ac1509b
a13342a
d00429b
da91ebc
3301a06
44df8ce
8972bbe
90df632
522650c
fb6a60f
2bf4719
3af0ab8
4ea36e8
1f9242f
e87a6b6
bac81af
6faabf9
2dcba3a
f3cc6b0
bdb2f26
43965be
9381644
c8a6c72
99f0e53
a4e05d1
a2bfaff
d91aab9
9835c81
5e51a3c
0b181fe
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -2,22 +2,92 @@ | |
* Appveyor-CI `win` builds [![AppVeyor Build status](https://ci.appveyor.com/api/projects/status/fyjgl66t943tf2yg/branch/master?svg=true)](https://ci.appveyor.com/project/jchodera/conda-recipes/branch/master) | ||
|
||
|
||
omnia-md/conda-recipes | ||
---------------------- | ||
# omnia-md/conda-recipes | ||
|
||
The recipes here create conda packages for scientific and numerical software components associated with the [`omnia`](http://omnia.md) project. | ||
The packages built from these recipes are shared with the community on [anaconda.org](https://anaconda.org/omnia). | ||
The packages built from these recipes are shared with the community on [anaconda.org](https://anaconda.org/omnia). | ||
These packages also depend on the `conda-forge` conda channel. | ||
|
||
### Installing packages from omnia | ||
## Installing packages from omnia | ||
|
||
To install a package | ||
To install a package (`mdtraj` for example) | ||
```bash | ||
# Set the channel priority behavior | ||
conda config --set channel_priority false | ||
# Add conda-forge as the lower priority (FILO format for priority) | ||
conda config --add channels conda-forge | ||
# Add the omnia channel | ||
$ conda config --add channels omnia | ||
conda config --add channels omnia | ||
# Install the 'mdtraj' a package | ||
$ conda install mdtraj | ||
conda install mdtraj | ||
``` | ||
|
||
The `channel_priority` behavior we configure gives priority to the highest version of the package over all channels. | ||
|
||
The default behavior (`channel_priority true`) pulls packages from the highest priority chanel, independent of package | ||
version numbers. i.e. an older version package on a higher priority channel will be installed over a newer version | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Remove "i.e." and capitalize the sentence. |
||
of the same package on a lower priority channel. | ||
|
||
### Important! | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Change "Important!" to "Important: Channel priority behavior must be disabled for omnia to work" |
||
|
||
These packages must use the old style of conda channel priority until omnia is entierly on conda-forge! | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. entirely |
||
The instructions below enforce the old channel priority format, please visit | ||
[the conda documentations on channel priority](https://conda.io/docs/channels.html) | ||
for more information. | ||
|
||
## Migration to conda-forge | ||
The Omnia project has started migrating to [`conda-forge`](https://conda-forge.github.io/). New packages that | ||
do not depend on OpenMM should be developed on `conda-forge` and exiting packages which do not depend on OpenMM | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. "exiting" -> "existing" |
||
should start migrating if possible. | ||
|
||
### Planed Migration Stages | ||
|
||
1. (**Current**) Update build image to CentOS 6 from CentOS 5 | ||
|
||
The base Docker image for linux builds will be updated to CentOS 6 with its new glibc. The base image is the | ||
`conda-forge` anvil, with some [custom addons](https://hub.docker.com/r/jchodera/omnia-linux-anvil/~/dockerfile/) | ||
to include things like the AMD SDK, TexLive, and CUDA for GPU builds. The updated version will ensure packages | ||
can work on the `conda-forge` platform which is CentOS 6 based. | ||
|
||
1. Stop building packages that appear in `conda-forge` | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. "For packages that appear in |
||
|
||
We want to minimize the amount of work we have to do as maintainers. To that end, we will stop building things | ||
which freely appear on `conda-forge` and maintained by someone other than us! | ||
For reproducibility purposes, we will keep our previously compiled versions, but they will not longer be updated. | ||
|
||
1. Begin migration of "nearly-pure Python" and non-OpenMM dependent packages | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Allow recipes that do not depend on OpenMM to migrate from |
||
|
||
* Packages which do not depend on OpenMM and can be run on CPUs only should start migrating over to `conda-forge` | ||
in preparation for the total migration. | ||
* Packages which can compile with just the `conda-forge` linux-anvil should also start migrating. | ||
* *We highly encourage devs of individual packages to start migrating now.* | ||
* **Once a package is on conda-forge, it should no longer depend packages from omnia!** | ||
|
||
1. Begin migration of `omnia` dependent (but not OpenMM dependent) packages to `conda-forge` | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not sure this stage is meaningful. We don't want users to migrate recipes that depend on other omnia recipes until their dependencies have migrated over---do we? |
||
|
||
* As more of the `omnia` dependencies appear on `conda-forge`, more packages can start moving over. | ||
* This stage can | ||
|
||
1. Determine the appropriate way to build packages which require more than the `conda-forge` linux-anvil can provide | ||
|
||
* The `conda-forge` linux-anvil does not support some things such as some LaTeX packages, AMD SDK, and CUDA files. | ||
* We will need to reach out to the conda-forge people to see what the best course of action is | ||
|
||
1. Move OpenMM to `conda-forge` | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. "Migrate OpenMM to |
||
|
||
* This will highly depend on the previous point and will probably have the longest time frame. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Just say "This requires us to identify the best way to add CUDA and AMD APP SDK developer tools to the |
||
|
||
1. Move packages which depend on OpenMM to `conda-forge` | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. "Migrate remaining packages to This can be combined with the final "Finish the move" |
||
|
||
1. Finish the move of all packages to `conda-forge` | ||
* Also ensure that all former omnia packages can be installed without the `omnia` conda channel | ||
|
||
1. Change this repo into an archive for reproducibility. | ||
|
||
### How to migrate to `conda-forge` (for existing packages) | ||
|
||
PLACEHOLDER | ||
|
||
### Supported versions | ||
|
||
Python packages are built against latest two releases of python (3.5 and 3.6) and python 2.7. | ||
|
@@ -31,16 +101,19 @@ Packages which have a binary dependency on [numpy](http://www.numpy.org/) are bu | |
|
||
The recipes here are automatically built using [Travis-CI](https://travis-ci.org/) for `linux` and `osx` and [Appveyor-CI](http://www.appveyor.com/) for `win`. | ||
|
||
For `linux` builds, we use a modified version of the [Holy Build Box](http://phusion.github.io/holy-build-box/), available [here](https://github.com/omnia-md/omnia-build-box), to ensure that the packages are fully compatible across multiple linux distributions and versions. | ||
For `linux` builds, we use a modified version of the | ||
[conda-forge linux-anvil](https://github.com/omnia-md/omnia-linux-anvil/blob/master/Dockerfile), | ||
to ensure that the packages are fully compatible across multiple linux distributions and versions. | ||
This build image contains the additional tools: | ||
* [clang](http://clang.llvm.org/) 3.8.1 | ||
* [TeXLive](https://www.tug.org/texlive/) 2015 | ||
* [TeXLive](https://www.tug.org/texlive/) 2016 | ||
* The [CUDA](https://developer.nvidia.com/cuda-toolkit) Toolkit version 8.0 | ||
* The [AMD APP SDK](http://developer.amd.com/tools-and-sdks/opencl-zone/amd-accelerated-parallel-processing-app-sdk/) 3.0 | ||
|
||
There is an additional image which has [clang](http://clang.llvm.org/) 3.8.1 | ||
|
||
To build a package yourself, run `conda build <package_name>`, or `./conda-build-all ./*` to build multiple packages across each of the supported python/numpy configurations. | ||
|
||
### Contributing a recipe | ||
### Contributing a recipe (this has not been updated to reflect the conda-forge changes) | ||
|
||
1. Fork this repo | ||
2. Add your `conda` recipe for building your package `packagename` in a subdirectory called `packagename`. Feel free to use other recipes here as examples. | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -15,6 +15,7 @@ build: | |
number: 0 | ||
entry_points: | ||
- coverage = coverage.cmdline:main | ||
skip: True | ||
|
||
requirements: | ||
build: | ||
|
This file was deleted.
This file was deleted.
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
So this one may have a name colision with FFTW on conda-forge. It appears to be the same thing. | ||
|
||
People will probably need to rename their dependencies |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -9,6 +9,7 @@ source: | |
build: | ||
preserve_egg_dir: True | ||
number: 0 | ||
skip: True # [win] | ||
|
||
requirements: | ||
build: | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think this is true anymore, since we're using
channel_priority false
. Maybe combine the two channel add commands with a single?