Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create dedicated environment YML for Windows & reenable tests #4785

Merged
merged 2 commits into from
Jun 19, 2021
Merged

Create dedicated environment YML for Windows & reenable tests #4785

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
2b0c17a
Create dedicated environment YML for Windows
michaelosthege Jun 17, 2021
710db87
Reenable Windows test workflow
michaelosthege Jun 19, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions .github/workflows/windows.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ on:

jobs:
pytest:
if: false
strategy:
matrix:
os: [windows-latest]
Expand All @@ -32,7 +31,7 @@ jobs:
with:
path: ~/conda_pkgs_dir
key: ${{ runner.os }}-conda-${{ env.CACHE_NUMBER }}-${{
hashFiles('conda-envs/environment-dev-py38.yml') }}
hashFiles('conda-envs/windows-environment-dev-py38.yml') }}
- name: Cache multiple paths
uses: actions/cache@v2
env:
Expand All @@ -49,7 +48,7 @@ jobs:
with:
activate-environment: pymc3-dev-py38
channel-priority: strict
environment-file: conda-envs/environment-dev-py38.yml
environment-file: conda-envs/windows-environment-dev-py38.yml
use-only-tar-bz2: true # IMPORTANT: This needs to be set for caching to work properly!
- name: Install-pymc3
run: |
Expand Down
37 changes: 37 additions & 0 deletions conda-envs/windows-environment-dev-py38.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
name: pymc3-dev-py38
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any particular reason the window dev environment needs to be different? If so add a comment up top so whoever reads this next is aware what those are.

I havet used windows in many years now so even I dont know, which is where the comment will help

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The m2w64-toolchain dependency is Windows-only, but conda doesn't support OS-conditional dependencies in these files..

I figured it was easier to maintain & diagnose two separate files than to find the set of dependencies that magically makes the conda dependency resolver find the correct result on all platforms.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add this explanation to the file

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deferred to #4786

channels:
- conda-forge
- defaults
dependencies:
# base dependencies (see install guide for Windows)
- aesara>=2.0.9
- arviz>=0.11.2
- cachetools>=4.2.1
- dill
- fastprogress>=0.2.0
- h5py>=2.7
- libpython
- mkl-service
- m2w64-toolchain
- numba
- numpy>=1.15.0
- pandas>=0.24.0
- pip
- python=3.8
- python-graphviz
- scipy>1.4.1
- typing-extensions>=3.7.4
# Extra stuff for dev, testing and docs build
- ipython>=7.16
- myst-nb
- nbsphinx>=0.4
- numpydoc>=0.9
- pre-commit>=2.8.0
- pydata-sphinx-theme
- pytest-cov>=2.5
- pytest>=3.0
- recommonmark>=0.4
- sphinx-autobuild>=0.7
- sphinx-panels
- sphinx>=1.5
- watermark
14 changes: 12 additions & 2 deletions scripts/generate_pip_deps_from_conda.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,17 @@

import yaml

EXCLUDE = {"python", "libblas", "mkl-service", "python-graphviz"}
EXCLUDE = {
"pip",
"python",
"libblas",
"libpython",
"m2w64-toolchain",
"mkl-service",
"numba",
"scipy",
"python-graphviz",
}
RENAME = {}


Expand Down Expand Up @@ -117,7 +127,7 @@ def main(conda_fname, pip_fname):
f"# This file is auto-generated by scripts/generate_pip_deps_from_conda.py, "
"do not modify.\n# See that file for comments about the need/usage of each dependency.\n\n"
)
pip_content = header + "\n".join(pip_deps) + "\n"
pip_content = header + "\n".join(sorted(pip_deps)) + "\n"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does it have to be sorted?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the Windows env file I split the dependencies by absolutely necessary vs. dev/docs related. The function here then took them in the same order, but requirements-dev.txt is sorted by another step in the precommit..


with open(pip_fname, "w") as pip_fd:
pip_fd.write(pip_content)
Expand Down