Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update to conda-build 3.10.9 #312

Merged
merged 19 commits into from
Jun 26, 2018
Merged

Conversation

mbargull
Copy link
Member

(based on gh-311)
Trying out conda-build 3.10.9 which includes a couple of linting speedups (such that a full linting job should be fast again -- only one remaining speedup issue is remains; see that issue for run time numbers).
Probably requires some compatibility fixes.

@johanneskoester johanneskoester merged commit 4f6b405 into cb3-migration Jun 26, 2018
@johanneskoester johanneskoester deleted the conda-build-3.10.9 branch June 26, 2018 17:32
@johanneskoester johanneskoester changed the title [WIP] update to conda-build 3.10.9 update to conda-build 3.10.9 Jun 26, 2018
johanneskoester added a commit that referenced this pull request Jul 19, 2018
* Pin to conda-build 3.

* Depend on latest conda-forge-pinning package.

This will ensure that conda-build 3 is always using the pinnings defined in
https://github.com/conda-forge/conda-forge-pinning-feedstock/blob/master/recipe/conda_build_config.yaml
By contributing to this list, we can maintain compatibility with conda-forge.

* Try downgrading to last minor release of conda.

* Try using conda 4.3.

* Try 4.5.0

* Handle UnableToParse

* Remove galaxy-lib version restriction.

* adapt to conda 4.5

* use get_output_file_paths instead of get_output_file_path

* Add lint for deprecated numpy spec.

* Document new lint.

* register lint.

* fix tests

* fix lint

* adapt test cases and remove obsolete ones.

* Remove env_matrix.

* fix typo

* fixes and further cleanup

* fixes

* Fixes.

* fix typo

* fixes

* fixes

* fixes.

* fix DAG

* Linting now uses fully rendered recipes, and checks all combinations of the variant matrix and both linux and osx.

* fix conda execution in docker

* fix argument

* fixes.

* Adapt numpy x.x lint to new multi-metadata strategy.

* properly render metadata when finding built package paths.

* fixes.

* fixes

* fix test for rendering sandboxing

* disable skip-existing

* Use same build args for the copying as for building.

* add test for conda-forge pinning

* add ability to have separate bioconda pins

* pep8

* add some more docs on testing

* add bioconda conda_build_config.yaml

* add first metadata fix

* update tests for load_first_metadata

* first pass of cb3 migration docs

* port bioconductor_skeleton to use cb3 compilers; improve detection

* update bioconductor_skeleton test

* more cb3 docs

* add anaconda to channels

* add anaconda channel to circleci setup

* short-circuit filtering if skipped

* fix text to only check names

* add test for when CI=true

* add extra checks that built pkgs exist

* add test for cb3 outputs

* clean up cb3 output test; typos

* utils.Progress: remove unnecessary sleep

* pkg_test: use --involucro-path to prevent download in galaxy-lib

* remove anaconda channel

* simplify htslib in test

* add compiler test

* fix htslib max pin

* update docs to reflect run_exports

* Update cb3.rst

* update docs

* add lint functions for compilers and fn

* add linting docs

* bring back the noarch lint docs

* render with finalize=False where possible

* rename load_conda_config to load_conda_build_config

* use conda.exports (public API)

* pin conda-forge-pinning, update conda/conda-build pins

* lint: handle if source is a list

* adjust tests

* show test durations

* lint: use decorator instead explicit loops

* lint: fix decorator, pass through __name__ for registry

* lint: fix decorator, pass through __name__ for registry (2)

* run long-running tests separately

* docker_utils: use base image acc. to BIOCONDA_UTILS_TAG

* Remove a priori filtering. Instead filter when each recipe is considered.

* Naming.

* Skip filter tests for now.

* Disable DAG generation.

* Activate DAG calculation again (we cannot avoid it because we need the topological sorting).

* Fast loading of metadata for DAG.

* Only use host dependencies when building DAG.

* get_package_paths returns tuple

* utils.get_package_paths: only return paths

* requirements: update conda, conda-build, conda-forge-pinning

* use bypass_env_check=True when we use non-finalized renderings

* requirements: use conda-build=3.10.3 for now

* test_utils: disable the remaining filter_recipes test

* Abort building if build strings in repo are divergent

* fix previous commit: add Python's redundant colon

* fix generate_docs.py

* fix another typo

* support linting just before building

* sort recipes for linting

* better noarch detection

* default df

* be clear which recipe the problem is on

* add back in debug info on linting

* more generic check for compilers

* add conda_build_config files consistently

* fix previous commit

* fix previous commit again

* fix mulled upload: iterate over all generated images.

* build: fix collecting mulled_images

* requirements: require python >=3.6

* build: skip builds early based on num of existing builds in channel

* build: topo-sort connected components before merging

* docs: handle extra.notes being a list

* build: handle UnsatisfiableError

* fix import

* [WIP] update to conda-build 3.10.9 (#312)

* build: rename --prelint to --lint, add --lint-only/--lint-exclude args

* requirements: update to conda-build 3.10.9

* build: remove default value for --lint-exclude

* requirements: require python >=3.6

* meta.get_section might return None (conda-build 3.10.9 compat)

* silence most flake8 nagging

* bioconductor: fix missing newline at EOF for post-link.sh

* fix typo

* use meta.get_value preferably

* make flake8 shut up already

* meta.get_value only works for path with depth 2

* lint: fix should_not_be_noarch for conda_build >3.10.3

* lint._has_compilers: also check for old compiler packages

* lint._get_deps: fix sections being unset on default

* docs: use only one job for debugging

* Add paper to the homepage.

* Handle dependency cycles (#313)

* fix minor bug in `build --lint-exclude`

* utils.check_recipe_skippable: avoid additional FILTER msg if nothing to do

* utils.get_dag: 'dep' can be empty string for unevaluated jinja expressions

* utils.get_dag: respect dependencies from build and run environments

* build.build_recipes: gracefully handle(=skip) packages in dependency cycles

* requirements: update to conda-forge-pinning 2018.07.18
johanneskoester pushed a commit that referenced this pull request Jul 20, 2018
* Pin to conda-build 3.

* Depend on latest conda-forge-pinning package.

This will ensure that conda-build 3 is always using the pinnings defined in
https://github.com/conda-forge/conda-forge-pinning-feedstock/blob/master/recipe/conda_build_config.yaml
By contributing to this list, we can maintain compatibility with conda-forge.

* Try downgrading to last minor release of conda.

* Try using conda 4.3.

* Try 4.5.0

* Handle UnableToParse

* Remove galaxy-lib version restriction.

* adapt to conda 4.5

* use get_output_file_paths instead of get_output_file_path

* Add lint for deprecated numpy spec.

* Document new lint.

* register lint.

* fix tests

* fix lint

* adapt test cases and remove obsolete ones.

* Remove env_matrix.

* fix typo

* fixes and further cleanup

* fixes

* Fixes.

* fix typo

* fixes

* fixes

* fixes.

* fix DAG

* Linting now uses fully rendered recipes, and checks all combinations of the variant matrix and both linux and osx.

* fix conda execution in docker

* fix argument

* fixes.

* Adapt numpy x.x lint to new multi-metadata strategy.

* properly render metadata when finding built package paths.

* fixes.

* fixes

* fix test for rendering sandboxing

* disable skip-existing

* Use same build args for the copying as for building.

* add test for conda-forge pinning

* add ability to have separate bioconda pins

* pep8

* add some more docs on testing

* add bioconda conda_build_config.yaml

* add first metadata fix

* update tests for load_first_metadata

* first pass of cb3 migration docs

* port bioconductor_skeleton to use cb3 compilers; improve detection

* update bioconductor_skeleton test

* more cb3 docs

* add anaconda to channels

* add anaconda channel to circleci setup

* short-circuit filtering if skipped

* fix text to only check names

* add test for when CI=true

* add extra checks that built pkgs exist

* add test for cb3 outputs

* clean up cb3 output test; typos

* utils.Progress: remove unnecessary sleep

* pkg_test: use --involucro-path to prevent download in galaxy-lib

* remove anaconda channel

* simplify htslib in test

* add compiler test

* fix htslib max pin

* update docs to reflect run_exports

* Update cb3.rst

* update docs

* add lint functions for compilers and fn

* add linting docs

* bring back the noarch lint docs

* render with finalize=False where possible

* rename load_conda_config to load_conda_build_config

* use conda.exports (public API)

* pin conda-forge-pinning, update conda/conda-build pins

* lint: handle if source is a list

* adjust tests

* show test durations

* lint: use decorator instead explicit loops

* lint: fix decorator, pass through __name__ for registry

* lint: fix decorator, pass through __name__ for registry (2)

* run long-running tests separately

* docker_utils: use base image acc. to BIOCONDA_UTILS_TAG

* Remove a priori filtering. Instead filter when each recipe is considered.

* Naming.

* Skip filter tests for now.

* Disable DAG generation.

* Activate DAG calculation again (we cannot avoid it because we need the topological sorting).

* Fast loading of metadata for DAG.

* Only use host dependencies when building DAG.

* get_package_paths returns tuple

* utils.get_package_paths: only return paths

* requirements: update conda, conda-build, conda-forge-pinning

* use bypass_env_check=True when we use non-finalized renderings

* requirements: use conda-build=3.10.3 for now

* test_utils: disable the remaining filter_recipes test

* Abort building if build strings in repo are divergent

* fix previous commit: add Python's redundant colon

* fix generate_docs.py

* fix another typo

* support linting just before building

* sort recipes for linting

* better noarch detection

* default df

* be clear which recipe the problem is on

* add back in debug info on linting

* more generic check for compilers

* add conda_build_config files consistently

* fix previous commit

* fix previous commit again

* fix mulled upload: iterate over all generated images.

* build: fix collecting mulled_images

* requirements: require python >=3.6

* build: skip builds early based on num of existing builds in channel

* build: topo-sort connected components before merging

* docs: handle extra.notes being a list

* build: handle UnsatisfiableError

* fix import

* [WIP] update to conda-build 3.10.9 (#312)

* build: rename --prelint to --lint, add --lint-only/--lint-exclude args

* requirements: update to conda-build 3.10.9

* build: remove default value for --lint-exclude

* requirements: require python >=3.6

* meta.get_section might return None (conda-build 3.10.9 compat)

* silence most flake8 nagging

* bioconductor: fix missing newline at EOF for post-link.sh

* fix typo

* use meta.get_value preferably

* make flake8 shut up already

* meta.get_value only works for path with depth 2

* lint: fix should_not_be_noarch for conda_build >3.10.3

* lint._has_compilers: also check for old compiler packages

* lint._get_deps: fix sections being unset on default

* docs: use only one job for debugging

* Add nested recipe check to get_recipes

Currently get_recipes only support recipes nested up to two deep from the recipe_folder provided. Our application of get_recipes, and potential future uses from others, has recipes nested deeper. This purposed implementation will provide the ability to locate all recipes, no matter how deep, within the user provided recipe_folder.

* Add test for new get_recipes deep nesting check

get_recipes was changed to allow for different levels of recipe nesting. test_depthcheck_get_recipe() will test get_recipes ability to find different levels of nested recipes.

* Update test_utils.py

* update test_depthcheck_get_recipe

* update test_depthcheck_get_recipe

* Update test_utils.py

* Update test_utils.py

* Update test_utils.py

* update test_depthcheck_get_recipe

* Re-test with circle ci

* Update utils.py

* Update test_utils.py

* Update test_utils.py

* Update test_utils.py

* Update utils.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants