Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BUILD: pandas on conda main repo installs numexpr, while on conda-forge it does not #59763

Open
1 task done
Dr-Irv opened this issue Sep 9, 2024 · 7 comments · May be fixed by #59822
Open
1 task done

BUILD: pandas on conda main repo installs numexpr, while on conda-forge it does not #59763

Dr-Irv opened this issue Sep 9, 2024 · 7 comments · May be fixed by #59822
Labels
Docs Upstream issue Issue related to pandas dependency

Comments

@Dr-Irv
Copy link
Contributor

Dr-Irv commented Sep 9, 2024

Installation check

Platform

Windows-11-10.0.22631-SP0

Installation Method

conda install

pandas Version

2.2.2

Python Version

3.12 and 3.11

Installation Logs

When you do conda install pandas and it uses the main repo, numexpr is automatically installed, even though we document it as being optional:

conda install pandas
Channels:
 - defaults
 - gurobi
Platform: win-64
Collecting package metadata (repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: C:\Condadirs\envs\tnumexpr

  added / updated specs:
    - pandas


The following NEW packages will be INSTALLED:

  blas               pkgs/main/win-64::blas-1.0-mkl
  bottleneck         pkgs/main/win-64::bottleneck-1.3.7-py311hd7041d2_0
  intel-openmp       pkgs/main/win-64::intel-openmp-2023.1.0-h59b6b97_46320
  mkl                pkgs/main/win-64::mkl-2023.1.0-h6b88ed4_46358
  mkl-service        pkgs/main/win-64::mkl-service-2.4.0-py311h2bbff1b_1
  mkl_fft            pkgs/main/win-64::mkl_fft-1.3.10-py311h827c3e9_0
  mkl_random         pkgs/main/win-64::mkl_random-1.2.7-py311hea22821_0
  numexpr            pkgs/main/win-64::numexpr-2.8.7-py311h1fcbade_0
  numpy              pkgs/main/win-64::numpy-1.26.4-py311hdab7c0b_0
  numpy-base         pkgs/main/win-64::numpy-base-1.26.4-py311hd01c5d8_0
  pandas             pkgs/main/win-64::pandas-2.2.2-py311hea22821_0
  python-dateutil    pkgs/main/win-64::python-dateutil-2.9.0post0-py311haa95532
  python-tzdata      pkgs/main/noarch::python-tzdata-2023.3-pyhd3eb1b0_0
  pytz               pkgs/main/win-64::pytz-2024.1-py311haa95532_0
  six                pkgs/main/noarch::six-1.16.0-pyhd3eb1b0_1
  tbb                pkgs/main/win-64::tbb-2021.8.0-h59b6b97_0


Proceed ([y]/n)? n

When you install pandas from conda-forge, numexpr is not included:

conda install pandas -c conda-forge
Channels:
 - conda-forge
 - defaults
 - gurobi
Platform: win-64
Collecting package metadata (repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: C:\Condadirs\envs\tnumexpr

  added / updated specs:
    - pandas


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    intel-openmp-2024.2.1      |    h57928b3_1083         1.8 MB  conda-forge
    libblas-3.9.0              |     23_win64_mkl         5.0 MB  conda-forge
    libcblas-3.9.0             |     23_win64_mkl         5.0 MB  conda-forge
    libexpat-2.6.3             |       he0c23c2_0         136 KB  conda-forge
    libhwloc-2.11.1            |default_h8125262_1000         2.3 MB  conda-fore
    liblapack-3.9.0            |     23_win64_mkl         5.0 MB  conda-forge
    libsqlite-3.46.1           |       h2466b09_0         856 KB  conda-forge
    mkl-2024.1.0               |     h66d3029_694       104.3 MB  conda-forge
    numpy-2.1.1                |  py312h49bc9c5_0         6.7 MB  conda-forge
    pandas-2.2.2               |  py312h72972c8_1        13.5 MB  conda-forge
    pthreads-win32-2.9.1       |       hfa6e2cd_3         141 KB  conda-forge
    python-3.12.5              |h889d299_0_cpython        15.2 MB  conda-forge
    tbb-2021.13.0              |       hc790b64_0         148 KB  conda-forge
    ------------------------------------------------------------
                                           Total:       159.8 MB

The following NEW packages will be INSTALLED:

  intel-openmp       conda-forge/win-64::intel-openmp-2024.2.1-h57928b3_1083
  libblas            conda-forge/win-64::libblas-3.9.0-23_win64_mkl
  libcblas           conda-forge/win-64::libcblas-3.9.0-23_win64_mkl
  libexpat           conda-forge/win-64::libexpat-2.6.3-he0c23c2_0
  libhwloc           conda-forge/win-64::libhwloc-2.11.1-default_h8125262_1000
  libiconv           conda-forge/win-64::libiconv-1.17-hcfcfb64_2
  liblapack          conda-forge/win-64::liblapack-3.9.0-23_win64_mkl
  libsqlite          conda-forge/win-64::libsqlite-3.46.1-h2466b09_0
  libxml2            conda-forge/win-64::libxml2-2.12.7-h0f24e4e_4
  libzlib            conda-forge/win-64::libzlib-1.3.1-h2466b09_1
  mkl                conda-forge/win-64::mkl-2024.1.0-h66d3029_694
  numpy              conda-forge/win-64::numpy-2.1.1-py312h49bc9c5_0
  pandas             conda-forge/win-64::pandas-2.2.2-py312h72972c8_1
  pthreads-win32     conda-forge/win-64::pthreads-win32-2.9.1-hfa6e2cd_3
  python-dateutil    conda-forge/noarch::python-dateutil-2.9.0-pyhd8ed1ab_0
  python-tzdata      conda-forge/noarch::python-tzdata-2024.1-pyhd8ed1ab_0
  python_abi         conda-forge/win-64::python_abi-3.12-5_cp312
  pytz               conda-forge/noarch::pytz-2024.1-pyhd8ed1ab_0
  six                conda-forge/noarch::six-1.16.0-pyh6c4a22f_0
  tbb                conda-forge/win-64::tbb-2021.13.0-hc790b64_0
  ucrt               conda-forge/win-64::ucrt-10.0.22621.0-h57928b3_0
  vc14_runtime       conda-forge/win-64::vc14_runtime-14.40.33810-hcc2c482_20

Are we OK with this difference? @lithomas1
@Dr-Irv Dr-Irv added Build Library building on various platforms Needs Triage Issue that has not been reviewed by a pandas team member labels Sep 9, 2024
@mroeschke
Copy link
Member

Since Anaconda controls the default channel, I suppose they have the right to include our "recommended" optional dependencies. IMO I think we are right not to include them through conda-forge as they are truly optional.

@Dr-Irv
Copy link
Contributor Author

Dr-Irv commented Sep 9, 2024

Since Anaconda controls the default channel, I suppose they have the right to include our "recommended" optional dependencies. IMO I think we are right not to include them through conda-forge as they are truly optional.

Well, yes, but do we need to document this?

I had some code that had installed pandas 2.2.2 from conda-forge, and got a new laptop, re-created the environment, which installed pandas from the default channel, and then I received all these numexpr warnings which didn't appear any more, so it was pretty confusing.

@lithomas1
Copy link
Member

I think I agree with Matt here.

You can file an issue with anaconda here maybe if you want them to drop numexpr as a dep from their feedstock
https://github.com/AnacondaRecipes/pandas-feedstock/issues

@jorisvandenbossche
Copy link
Member

I think it would be good to raise an issue about this in the AnacondaRecipes feedstock

@Dr-Irv
Copy link
Contributor Author

Dr-Irv commented Sep 9, 2024

The other option is that we change our documentation to reflect that installations from conda-forge (which we document) and installations from the main channel could be different.

@mroeschke mroeschke added Docs and removed Build Library building on various platforms Needs Triage Issue that has not been reviewed by a pandas team member labels Sep 9, 2024
@lithomas1
Copy link
Member

The other option is that we change our documentation to reflect that installations from conda-forge (which we document) and installations from the main channel could be different.

Let's see what the Anaconda folks think first.
numexpr maintenance has been spotty a bit now, so it's possible we can convince them to drop the dep.

@lithomas1 lithomas1 added the Upstream issue Issue related to pandas dependency label Sep 9, 2024
@Dr-Irv Dr-Irv mentioned this issue Sep 10, 2024
Open
@Dr-Irv
Copy link
Contributor Author

Dr-Irv commented Sep 10, 2024

Let's see what the Anaconda folks think first. numexpr maintenance has been spotty a bit now, so it's possible we can convince them to drop the dep.

Created AnacondaRecipes/pandas-feedstock#45

@mroeschke mroeschke linked a pull request Sep 16, 2024 that will close this issue
Open
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Docs Upstream issue Issue related to pandas dependency
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

DOC: Emphasize team managed pandas in installation docs mroeschke/pandas
4 participants
@jorisvandenbossche @mroeschke @Dr-Irv @lithomas1