[ci] Install graphviz system-widely #4238

StrikerRUS · 2021-04-28T13:20:02Z

graphviz cannot be installed from the default conda channel with Python 3.9:

UnsatisfiableError: The following specifications were found
to be incompatible with the existing python installation in your environment:

Specifications:

  - python-graphviz -> python[version='>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.8,<3.9.0a0|>=3.7,<3.8.0a0|>=3.6,<3.7.0a0']

Your python: python=3.9

If python is on the left-most side of the chain, that's the version you've asked for.
When python appears to the right, that indicates that the thing on the left is somehow
not available for the python version you are constrained to. Note that conda will not
change your python version to a different minor version unless you explicitly specify
that.

The following specifications were found to be incompatible with each other:

Output in format: Requested package -> Available versions

Package six conflicts for:
scikit-learn -> mkl-service[version='>=2,<3.0a0'] -> six
dask -> bokeh[version='>=1.0.0,!=2.0.0'] -> six[version='>=1.5.2']
distributed -> six
matplotlib -> cycler[version='>=0.10'] -> six[version='>=1.5']
scipy -> mkl-service[version='>=2.3.0,<3.0a0'] -> six
pytest -> more-itertools[version='>=4.0.0'] -> six[version='>=1.0.0,<2.0.0']
pytest -> six[version='>=1.10.0']
pandas -> python-dateutil[version='>=2.7.3'] -> six[version='>=1.5']
numpy -> mkl-service[version='>=2.3.0,<3.0a0'] -> six

Package libcxxabi conflicts for:
scikit-learn -> libcxx[version='>=4.0.1'] -> libcxxabi==4.0.1[build='hcfea43d_1|hebd6815_0']
pandas -> libcxx[version='>=4.0.1'] -> libcxxabi==4.0.1[build='hcfea43d_1|hebd6815_0']
scipy -> libcxx[version='>=4.0.1'] -> libcxxabi==4.0.1[build='hcfea43d_1|hebd6815_0']
matplotlib -> libcxx[version='>=4.0.1'] -> libcxxabi==4.0.1[build='hcfea43d_1|hebd6815_0']

Package pyparsing conflicts for:
pytest -> packaging -> pyparsing[version='>=2.0.2']
matplotlib -> matplotlib-base[version='>=3.3.4,<3.3.5.0a0'] -> pyparsing[version='>=2.0.3,!=2.0.4,!=2.1.2,!=2.1.6']
matplotlib -> pyparsing

Package psutil conflicts for:
distributed -> psutil[version='>=5.0']
dask -> distributed[version='>=2021.4.0'] -> psutil[version='>=5.0']

Package certifi conflicts for:
joblib -> setuptools -> certifi[version='>=2016.09|>=2016.9.26']
matplotlib -> matplotlib-base[version='>=3.3.4,<3.3.5.0a0'] -> certifi[version='>=2016.09|>=2016.9.26|>=2020.06.20']
pytest -> setuptools[version='>=40.0'] -> certifi[version='>=2016.09|>=2016.9.26']
distributed -> setuptools -> certifi[version='>=2016.09|>=2016.9.26']

Package pandas conflicts for:
dask -> bokeh -> pandas
dask -> pandas[version='>=0.19.0|>=0.21.0|>=0.23.0|>=0.25.0']

Package intel-openmp conflicts for:
scipy -> mkl[version='>=2021.2.0,<2022.0a0'] -> intel-openmp=2021
scikit-learn -> mkl[version='>=2019.4,<2020.0a0'] -> intel-openmp
numpy -> mkl[version='>=2021.2.0,<2022.0a0'] -> intel-openmp=2021

Package numpy conflicts for:
dask -> numpy[version='>=1.10|>=1.10.4|>=1.11.0|>=1.13.0|>=1.15.1|>=1.16']
matplotlib -> numpy[version='>=1.14.6,<2.0a0']
matplotlib -> matplotlib-base[version='>=3.3.4,<3.3.5.0a0'] -> numpy[version='>=1.15.4,<2.0a0|>=1.16.6,<2.0a0']
scikit-learn -> numpy[version='>=1.11.3,<2.0a0|>=1.14.6,<2.0a0|>=1.16.6,<2.0a0|>=1.9.3,<2.0a0']
dask -> bokeh[version='>=1.0.0,!=2.0.0'] -> numpy[version='>=1.11.3,<2.0a0|>=1.11.3|>=1.7.1|>=1.19.2,<2.0a0|>=1.16.6,<2.0a0|>=1.15.4,<2.0a0|>=1.14.6,<2.0a0|>=1.13.3,<2.0a0|>=1.12.1,<2.0a0|>=1.9.3,<2.0a0|>=1.9.3,<1.10.0a0|>=1.9']
pandas -> numpy[version='>=1.11.3,<2.0a0|>=1.12.1,<2.0a0|>=1.13.3,<2.0a0|>=1.14.6,<2.0a0|>=1.15.4,<2.0a0|>=1.16.6,<2.0a0|>=1.19.2,<2.0a0|>=1.9.3,<2.0a0|>=1.9.3,<1.10.0a0|>=1.9']
scipy -> numpy[version='>=1.11.3,<2.0a0|>=1.14.6,<2.0a0|>=1.16.6,<2.0a0|>=1.15.1,<2.0a0|>=1.9.3,<2.0a0']
scikit-learn -> scipy -> numpy[version='>=1.15.1,<2.0a0']

Package freetype conflicts for:
matplotlib -> freetype[version='>=2.8,<2.9.0a0|>=2.9.1,<3.0a0']
matplotlib -> matplotlib-base[version='>=3.3.4,<3.3.5.0a0'] -> freetype[version='>=2.10.2,<3.0a0|>=2.10.4,<3.0a0']
python-graphviz -> graphviz -> freetype[version='>=2.8,<2.9.0a0|>=2.9.1,<3.0a0']

Package backports_abc conflicts for:
distributed -> tornado[version='>=5'] -> backports_abc[version='>=0.4']
matplotlib -> tornado -> backports_abc[version='>=0.4']

Package packaging conflicts for:
dask -> bokeh[version='>=1.0.0,!=2.0.0'] -> packaging[version='>=16.8']
pytest -> packaging

Package cloudpickle conflicts for:
distributed -> dask-core[version='>=2020.12.0'] -> cloudpickle[version='>=1.1.1']
dask -> cloudpickle[version='>=0.2.1|>=0.2.2']
distributed -> cloudpickle[version='>=0.2.2|>=1.3.0|>=1.5.0']
dask -> dask-core=2021.4.0 -> cloudpickle[version='>=1.1.1|>=1.5.0|>=1.3.0']

Package setuptools conflicts for:
pytest -> setuptools[version='>=40.0']
python=3.9 -> pip -> setuptools
dask -> distributed[version='>=2021.4.0'] -> setuptools
distributed -> setuptools
matplotlib -> setuptools
joblib -> setuptools
scikit-learn -> joblib[version='>=0.11'] -> setuptools

Package python-dateutil conflicts for:
dask -> bokeh[version='>=1.0.0,!=2.0.0'] -> python-dateutil[version='>=2.1|>=2.7.3|>=2.6.1|>=2.5.*']
matplotlib -> python-dateutil
matplotlib -> matplotlib-base[version='>=3.3.4,<3.3.5.0a0'] -> python-dateutil[version='>=2.1']
pandas -> python-dateutil[version='>=2.5.*|>=2.6.1|>=2.7.3']

Package singledispatch conflicts for:
matplotlib -> tornado -> singledispatch
dask -> distributed[version='>=1.28.0'] -> singledispatch
distributed -> singledispatch

so install it via apt-get/yum/brew.

sudo add-apt-repository -y ppa:git-core/ppa

is not needed in the latest Ubuntu - default git is quite OK.

jameslamb

Great, thanks for figuring this out!

I think the conda conflicts issue (#4204) is the second-most-common CI failure I've seen recently (#3817 is the one I see most frequently). So I'm very glad to have this change.

jameslamb · 2021-04-28T22:10:35Z

.ci/setup.sh

@@ -25,13 +26,10 @@ else  # Linux
        echo 'debconf debconf/frontend select Noninteractive' | sudo debconf-set-selections

        sudo apt-get update
-        sudo apt-get install -y --no-install-recommends \
+        sudo apt-get install --no-install-recommends -y \


For my own understanding...does the order of these arguments change the behavior of this call? Or are you just taking this opportunity to standardize all the apt-get install calls?

I'm totally ok with this change being part of this PR, just want to understand.

I'm

taking this opportunity to standardize all the apt-get install calls

🙂

…4097, #4238)

* [ci] use conda-forge in CI jobs (fixes #4948) * comment out more jobs * try reverting graphviz patch, running more cuda jobs * get graphviz from PyPI and try removing some patches for r-lintr * start running appveyor again * use conda-forge if using conda * fix commands * conda install graphviz * try newer openmp * pin below openmp 11.x * focus on gpu task * trying to narrow down error * maybe gcc11 is the issue * start adding other tests back * pin openmp too * maybe need to pin to gcc less than 10.x * pin libgfortran and libstdcxx as well * pin to gcc 9.3.0 * move constraints up to initial environment * add all CI jobs back * try installing python-graphviz separately * try new lightgbm/vsts-agent image * fix typo * test if pinning gcc for linux gpu_source build is still necessary * ok yes, pinning gcc is necessary * test if Linux gpu_source works with Python 3.9.6 * no special exception for Linux gpu_source job * pin to Python 3.9.6 in Linux gpu_source * try explicitly asking for libstdcxx-ng for every linux build * swap compilers * switch compilers back * revert accidental whitespace change * comment out CI * try Linux gpu_source with different Python versions * Revert "try Linux gpu_source with different Python versions" This reverts commit f6f63cb. * Revert "comment out CI" This reverts commit ece191f. * remove libxml2 install, change CONDA path * avoid installing conda in rchk job * empty commit 1 * empty commit 2 * empty commit 3 * empty commit 4 * add more verbose logging around installation of python-graphviz * empty commit 1 * get mamba info * get more conda info * add another mamba info call * allow for other macOS environments in GHA configuration * Revert "allow for other macOS environments in GHA configuration" This reverts commit a3c7a19. * get more logs from mamba * get Build.ArtifactsStagingDirectory * get more logs and try to force re-installing everything * clean cache after every step * remove --update-all and make logs less verbose * remove more print statements and uncomment jobs * test if conda-clean issue fixes segfaults for gpu_source * pin python version for gpu_source * empty commit 1 * use miniforge instead * empty commit 1 * Apply suggestions from code review * bring workarounds back * remove duplicated graphviz system-wide installation (reverts #4095, #4097, #4238) * empty commit 1 * empty commit 2 * empty commit 3 * empty commit 4 * empty commit 5 * empty commit 6 * empty commit 7 * empty commit 8 * empty commit 9 * empty commit 10 * empty commit 10 * empty commit 10 * empty commit 10 * empty commit 11 * one more try * try to downgrade Python version for Linux GPU job * swap compilers * Revert "swap compilers" This reverts commit f04dc27. Co-authored-by: Nikita Titov <nekit94-12@hotmail.com> Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

github-actions · 2023-08-23T22:39:45Z

This pull request has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this.

StrikerRUS added 5 commits April 28, 2021 16:05

Install graphviz from default conda channel

b282dcb

Update test.sh

68e2030

Update setup.sh

a4bae7b

Update test.sh

c1643ea

Update setup.sh

1282ac1

StrikerRUS changed the title ~~[ci] Install graphviz from default conda channel~~ [ci] Install graphviz from apt-get and brew Apr 28, 2021

StrikerRUS added 4 commits April 28, 2021 19:20

Update setup.sh

fcb939c

Update setup.sh

f314c39

Update setup.sh

8bd980a

Update setup.sh

0243c09

StrikerRUS changed the title ~~[ci] Install graphviz from apt-get and brew~~ [ci] Install graphviz system-widely Apr 28, 2021

Update setup.sh

12e4389

StrikerRUS added the maintenance label Apr 28, 2021

StrikerRUS marked this pull request as ready for review April 28, 2021 20:47

StrikerRUS requested a review from jameslamb as a code owner April 28, 2021 20:47

jameslamb approved these changes Apr 28, 2021

View reviewed changes

StrikerRUS merged commit 91f72e2 into master Apr 29, 2021

StrikerRUS deleted the graphviz branch April 29, 2021 21:59

StrikerRUS added a commit that referenced this pull request Feb 4, 2022

remove duplicated graphviz system-wide installation (reverts #4095, #…

b33195e

…4097, #4238)

github-actions bot locked as resolved and limited conversation to collaborators Aug 23, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ci] Install graphviz system-widely #4238

[ci] Install graphviz system-widely #4238

StrikerRUS commented Apr 28, 2021 •

edited

Loading

jameslamb left a comment

jameslamb Apr 28, 2021

StrikerRUS Apr 29, 2021 •

edited

Loading

github-actions bot commented Aug 23, 2023

[ci] Install graphviz system-widely #4238

[ci] Install graphviz system-widely #4238

Conversation

StrikerRUS commented Apr 28, 2021 • edited Loading

jameslamb left a comment

Choose a reason for hiding this comment

jameslamb Apr 28, 2021

Choose a reason for hiding this comment

StrikerRUS Apr 29, 2021 • edited Loading

Choose a reason for hiding this comment

github-actions bot commented Aug 23, 2023

StrikerRUS commented Apr 28, 2021 •

edited

Loading

StrikerRUS Apr 29, 2021 •

edited

Loading