conda environment is not setting up expected env vars for gfortran

[DougBurke]

Details

• Read the Docs project URL: https://readthedocs.org/projects/sherpa-test/
• Build URL (if applicable): https://readthedocs.org/projects/sherpa-test/builds/7151907/

Expected Result

To build the package using the compilers provided by conda (in this case gfortran).

Actual Result

I want to build a Python package using conda and its version of gfortran - i.e. the gfortran_linux-64 package mentioned at https://conda.io/docs/user-guide/tasks/build-packages/compiler-tools.html - which, thanks to "reasons", is accessed via an environment variable (e.g. $GFORTRAN / $F77). These environment variables are set up automatically when you "activate" the conda environment, but it appears that this isn't happening. I have instrumented setup.py to display the environment variables just before calling setup, and the build log shows they are not being set.

I had a quick dig through the code but couldn't see how the conda environment was being entered/activated by the rtd builder.

The conda env create --name latest --file /home/docs/checkouts/readthedocs.org/user_builds/sherpa-test/checkouts/latest/docs/environment.yml shows that gfortran_linux-64 has been installed (and I have an old build at https://readthedocs.org/projects/sherpa-test/builds/7149148/ where I added in the output of find in the conda environment to show that the files are available, including the "activation scripts" that set the env. variables).

The start of the python setup.py install --force is shown below, and the DEBUG info comes from a simple hack which can be found at https://github.com/DougBurke/sherpa/blob/684885e159cc57854bb11578c820384b8863ee39/setup.py#L137

# conda environments:
#
latest                *  /home/docs/checkouts/readthedocs.org/user_builds/sherpa-test/conda/latest
root                     /home/docs/.conda

*** DEBUG start
  F77 -> undefined
  F90 -> undefined
  F95 -> undefined
  FC -> undefined
  FORTRAN -> undefined
  GFORTRAN -> undefined
--
--
--
*** DEBUG end
running sherpa_config
...

which shows that the latest conda environment is activated, but some of the env variables I would expect to be set (given that gfortran_linux-64 was installed) are not defined. I expected the env. variables in the DEBUG section above to be set to something like

GFORTRAN -> /home/docs/checkouts/readthedocs.org/user_builds/sherpa-test/conda/latest/bin/x86_64-conda_cos6-linux-gnu-gfortran

[DougBurke]

I edited the issue to include a link to the code that displays the environment variables.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[humitos]

We have a new feature that allow us to set env variables in case you still need them: https://docs.readthedocs.io/en/latest/guides/environment-variables.html

[ihnorton]

I need this as well. The problem seems to be that the conda environment is not activated before running setup.py (CONDA_PREFIX is not set). I could work around this in setup.py by conditioning on env['READTHEDOCS'], but I haven't yet found a reliable way to get the CONDA_PREFIX corresponding to the environment.

See an example build here: https://readthedocs.com/api/v2/build/199880.txt -- in the log there, I would like to be able to consistently find the path given by the conda status output as the line environment location: /home/docs/checkouts/readthedocs.org/user_builds/tiledb-inc-tiledb-py/conda/ihn-fix_docs, but within setup.py.

[wmay]

I experimented with this and saw that the path to the conda environment is /home/docs/checkouts/readthedocs.org/user_builds/<pkgname>/conda/latest/, and the path to the package source code is /home/docs/checkouts/readthedocs.org/user_builds/<pkgname>/checkouts/latest/.

I got gfortran to work by including this code near the top of my setup.py:

import os
if os.environ.get('READTHEDOCS') == 'True':
    # link to the conda fortran compiler for readthedocs
    file_path = os.path.realpath(__file__)
    checkout_name = os.path.basename(os.path.dirname(file_path))
    gfortran_rel_path = '../../conda/' + checkout_name + '/bin/x86_64-conda_cos6-linux-gnu-gfortran'
    gfortran_path = os.path.realpath(gfortran_rel_path)
    os.environ['F77'] = gfortran_path
    os.environ['F90'] = gfortran_path

As long as the conda environment name is the same as the name of the checked-out version of the code, it should provide the correct gfortran path.

[RaulPPelaez]

This is still an issue, I cannot put something like this in my .readthedocs.yaml file:

build:
  os: "ubuntu-22.04"
  tools:
     python: "mambaforge-22.9"
  jobs:
     post_create_environment:
       - cmake -DCMAKE_INSTALL_PREFIX=${CONDA_PREFIX} . && make -j4  all install

I managed to solve it by asking for the location of pip:

build:
  os: "ubuntu-22.04"
  tools:
     python: "mambaforge-22.9"
  jobs:
     post_create_environment:
       - cmake -DCMAKE_INSTALL_PREFIX=$(which pip | rev | cut -d/ -f3- | rev) . && make -j4  all install

Something like this could also be used, but have not tested:

conda info | grep "active env location" | cut -d: -f2

PS: I am resurrecting this issue because it is the top search for this.