Build Fails to Create Conda Environment #2651

swainn · 2017-02-18T03:46:35Z

Details

Project URL: https://readthedocs.org/projects/tethys-platform/
Build URL (if applicable): https://readthedocs.org/projects/tethys-platform/builds/5043136/
Read the Docs username (if applicable): swainn

Expected Result

I expect the docs for the dev branch of my project to build.

Actual Result

I encounter a failure during the creation of the conda environment (see full output below).

Things I have tried:

(1) Found this suggestion on the Conda issues page, so I attempted wiping the environment several times.
(2) Reverted the back to commit with last successful docs build, but still failed with same error.
(3) Created this Conda environment my dev machine successfully (using miniconda).

Here is the error:

conda env create --name dev --file /home/docs/checkouts/readthedocs.org/user_builds/tethys-platform/checkouts/dev/docs-conda_env.yml
Fetching package metadata .........
Solving package specifications: ..........
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
An unexpected error has occurred.
Please consider posting the following information to the
conda GitHub issue tracker at:

https://github.com/conda/conda/issues

Current conda install:

           platform : linux-64
      conda version : 4.2.12
   conda is private : False
  conda-env version : 4.2.12
conda-build version : not installed
     python version : 2.7.12.final.0
   requests version : 2.11.1
   root environment : /usr/local/miniconda  (read only)
default environment : /home/docs/checkouts/readthedocs.org/user_builds/tethys-platform/conda/dev
   envs directories : /home/docs/checkouts/readthedocs.org/user_builds/tethys-platform/conda
                      /home/docs/.conda/envs
                      /usr/local/miniconda/envs
      package cache : /home/docs/checkouts/readthedocs.org/user_builds/tethys-platform/conda/.pkgs
                      /home/docs/.conda/envs/.pkgs
                      /usr/local/miniconda/pkgs
       channel URLs : https://repo.continuum.io/pkgs/free/linux-64
                      https://repo.continuum.io/pkgs/free/noarch
                      https://repo.continuum.io/pkgs/pro/linux-64
                      https://repo.continuum.io/pkgs/pro/noarch
        config file : None
       offline mode : False

$ /usr/local/miniconda/bin/conda-env create --name dev --file /home/docs/checkouts/readthedocs.org/user_builds/tethys-platform/checkouts/dev/docs-conda_env.yml

Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python2.7/site-packages/conda/exceptions.py", line 479, in conda_exception_handler
    return_value = func(*args, **kwargs)
  File "/usr/local/miniconda/lib/python2.7/site-packages/conda_env/cli/main_create.py", line 111, in execute
    installer.install(prefix, pkg_specs, args, env)
  File "/usr/local/miniconda/lib/python2.7/site-packages/conda_env/installers/conda.py", line 38, in install
    raise CondaRuntimeError('RuntimeError: %s' % e)
CondaRuntimeError: Runtime error: RuntimeError: Runtime error: Could not open u'/usr/local/miniconda/pkgs/yaml-0.1.6-0.tar.bz2.part' for writing ([Errno 13] Permission denied: u'/usr/local/miniconda/pkgs/yaml-0.1.6-0.tar.bz2.part').

Fetching packages ...
yaml-0.1.6-0.t 0% | | ETA: --:--:-- 0.00 B/s

The text was updated successfully, but these errors were encountered:

fmaussion · 2017-02-21T09:08:55Z

Hi, we have the same problem since about 1 week:

xarray: https://readthedocs.org/projects/xray/builds/5051404/

oggm: https://readthedocs.org/projects/oggm/builds/5053357/

swainn · 2017-02-23T03:19:11Z

It seems this issue can be remedied by upgrading to conda 4.2.13. See: conda/conda#3448 (comment)

fmaussion · 2017-02-23T07:38:07Z

Thanks. Do we have any control on the conda version installed on RTD?

fmaussion · 2017-02-23T17:47:30Z

What is striking me is that we are the only ones reporting this. I'd expect such an issue to affect much more people?

woodmd · 2017-02-24T00:41:03Z

I have the same error:
https://readthedocs.org/projects/fermipy/builds/5067364/

fmaussion · 2017-02-28T13:45:12Z

Our docs aren't built since two weeks now... :-(

Did anyone come up with a solution?

swainn · 2017-02-28T17:35:30Z

Same here. We've had no docs builds succeed for nearly two weeks and no response from RTD...

spencerahill · 2017-03-02T23:11:32Z

Same problem for us too: https://readthedocs.org/projects/aospy/builds/5097720/

snowman2 · 2017-03-03T15:36:41Z

Similar issue here: http://readthedocs.org/projects/gsshapy/builds/5091350. Builds having issues for over 2 weeks.

dcolligan · 2017-03-03T19:39:45Z

Same issue on our builds, for the same period of time as others are reporting:

https://readthedocs.org/projects/ga4gh-reference-implementation/builds/5097461/
https://readthedocs.org/projects/ga4gh-client/builds/5097407/
https://readthedocs.org/projects/ga4gh-schemas/builds/5051360/

david4096 · 2017-03-07T16:56:10Z

We're considering dropping RTD because of this. Is there something we can do to help the update of conda happen on RTD?

humitos · 2017-03-07T17:18:30Z

@agjohnson now that we are talking about upgrading the packages from conda-forge in #2378, is it possible to upgrade the conda version also? Since it's a bugfix version I think there is no problem to upgrade from 4.2.12 to 4.2.13 (people says this problem is fixed there). What do you think?

Considering how RTD internally works with conda (https://github.com/rtfd/readthedocs-docker-images/blob/master/Dockerfile#L40), it seems that this upgrade won't be available until the next docker image is built and deployed to production.

Other (some of them, quick-and-dirty) solutions I have on the top of my head:

run conda update conda before each of the builds (kind of dangerous since RTD can't guarantee a specific version)
add a config in the yml to pin the conda version before running the build
upgrade the conda version just once inside the docker container used to build the documentation (hotfix)

The latest version of conda seems to be 4.3.14-py36_0 in my pc when running conda update conda

ericholscher · 2017-03-07T17:19:14Z

I don't fully understand conda, and we need people on our core team who can actually track and update the usage. What is the correct fix for this? Is it something that we broke, or did conda break?

ericholscher · 2017-03-07T17:19:50Z

I'm happy to upgrade conda if that is the correct fix -- but I'd like to know the root cause of what broke, so that we can prevent it from happening again.

snowman2 · 2017-03-07T17:24:41Z

Here is a discussion on the issue on conda: conda/conda#3448 (comment)

ericholscher · 2017-03-07T17:32:49Z

So this was an issue with the version of conda from when we created the last set of docker images? Is there a way to pin an install of conda to a specific verison? The official install docs only mention a way to install the "latest".

ericholscher · 2017-03-07T17:44:38Z

The current plan is to rebuild the 2.0 docker image, which I believe will cause it to install the latest version of conda. I'd like to figure out a way to pin the version to prevent this issue going forward. We've had a lot of breakage in the conda tooling specifically, and I'd like to knowingly upgrade it, instead of it happening randomly on image rebuilds.

ericholscher · 2017-03-07T17:52:58Z

Image rebuild is going here: https://hub.docker.com/r/readthedocs/build/builds/bgpgd7axkbaihm6wkexya6h/ -- it will probably take ~30 mins

humitos · 2017-03-07T17:53:44Z

I'd like to figure out a way to pin the version to prevent this issue going forward.

There is a blog post about that: http://www.damian.oquanta.info/posts/how-to-pin-conda.html

Anyway, it says that conda will upgrade automatically on each install (I just tried that at my pc and it's true) and if you want to not do so, you have to pin conda itself by using the pinned file (https://conda.io/docs/faq.html#pinning-packages)

[12:44] humitos: ~> conda install scipy
Fetching package metadata .........
Solving package specifications: .

Package plan for installation in environment /home/humitos/.miniconda3:

The following NEW packages will be INSTALLED:

    libgfortran: 3.0.0-1           
    mkl:         2017.0.1-0        
    numpy:       1.12.0-py36_0     
    scipy:       0.18.1-np112py36_1

The following packages will be UPDATED:

    conda:       4.3.11-py36_0      --> 4.3.14-py36_0

Proceed ([y]/n)?

On the other hand, I didn't find a way to install for the first time an specific version of conda. It seems that you have to install the latest one and then you can do conda update conda=x.x.x if you want to pin it.

ericholscher · 2017-03-07T17:54:35Z

Sounds like a way forward at least. So we can do the network install, and then do the conda update conda=x.x.x after that in the Dockerfile.

ericholscher · 2017-03-07T17:55:21Z

I'm not convinced that won't break at some point in the future, but at least it makes the builds more deterministic :)

snowman2 · 2017-03-07T18:01:41Z

Please don't deprecate conda support. Is makes using/distributing Python packages soo much easier.

agjohnson · 2017-03-07T18:25:32Z

Moving conversation around the Docker image to readthedocs/readthedocs-docker-images#22

A new build image likely won't hit our servers for a few hours, the build process takes a while.

Also, I'll say that for anyone wondering why their project is still hitting this issue, consider contributing your time to resolve this. We are community run service and need both your input and your contributions.

snowman2 · 2017-03-07T19:49:42Z

4.3.11 is the version failing on my current build and for @humitos. 4.2.12 failed for the original poster @swainn. So, maybe try 4.2.11 (2016-10-24).

humitos · 2017-03-07T19:54:49Z

wait, is the latest version of conda failing to create an environment and install some packages? Instead of us keep upgrading the RTD server to a random version of conda, can you (people with failing builds) test this locally by creating the proper conda environment with the different versions and confirm which version of conda is not having troubles in your pc?

If conda is broken there is nothing that we can do at RTD but just wait for a fix on them :)

ericholscher · 2017-03-07T19:56:53Z

Agreed -- I don't really know how to fix this, other than shooting in the dark at random released or unreleased versions of miniconda. I'd love to know a known good version, and be able to go with it moving forward.

humitos · 2017-03-07T20:04:38Z

This build (https://readthedocs.org/projects/aospy/builds/5117255/) has a different error and I'm guessing the Permission denied problem could be related because the miniconda is installed as root and then it's executed as docs:

Installed as root: https://github.com/rtfd/readthedocs-docker-images/blob/releases/2.x/Dockerfile#L37
User changed to docs: https://github.com/rtfd/readthedocs-docker-images/blob/releases/2.x/Dockerfile#L44

(the docker hub image build related: https://hub.docker.com/r/readthedocs/build/builds/bgpgd7axkbaihm6wkexya6h/)

agjohnson · 2017-03-07T20:14:06Z

If this is a permission issue: readthedocs/readthedocs-docker-images#24

Testing this shouldn't be difficult.

ericholscher · 2017-03-07T20:22:10Z

That doesn't explain why this recently started failing though?

agjohnson · 2017-03-07T20:29:44Z

There were two major changes in this timeframe: we rerolled the build images (upgrading conda), and the new image is based on ubuntu 16.04 now. I have no clue what the internals of the shell script do, but if it was doing something different in 14.04 vs 16.04, this could be the cause.

humitos · 2017-03-07T22:56:27Z

Just to add more context here. I tried to build one of the failing project outside RTD (env, docker img, etc) and it worked:

[14:59] humitos: repositories> git clone https://github.com/spencerahill/aospy.git
Clonar en «aospy»...
remote: Counting objects: 2156, done.
remote: Compressing objects: 100% (22/22), done.
remote: Total 2156 (delta 7), reused 1 (delta 1), pack-reused 2133
Receiving objects: 100% (2156/2156), 3.62 MiB | 484.00 KiB/s, done.
Resolving deltas: 100% (1468/1468), done.
[14:59] humitos: repositories> cd aospy/
/home/humitos/readthedocs/repositories/aospy
[14:59] humitos: aospy> conda env create --name aospy_latest --file docs/environment.yml 
Fetching package metadata ...........
Solving package specifications: .
ca-certificate 100% |##############################################################################################################################################| Time: 0:00:03  53.90 kB/s

[... snipped ...]

matplotlib-1.5 100% |##############################################################################################################################################| Time: 0:00:14 543.61 kB/s
#
# To activate this environment, use:
# > source activate aospy_latest
#
# To deactivate this environment, use:
# > source deactivate aospy_latest
#

[15:13] humitos: aospy> conda info
Current conda install:

               platform : linux-64
          conda version : 4.3.11
       conda is private : False
      conda-env version : 4.3.11
    conda-build version : not installed
         python version : 3.6.0.final.0
       requests version : 2.12.4
       root environment : /home/humitos/.miniconda3  (writable)
    default environment : /home/humitos/.miniconda3
       envs directories : /home/humitos/.miniconda3/envs
                          /home/humitos/.conda/envs
          package cache : /home/humitos/.miniconda3/pkgs
                          /home/humitos/.conda/pkgs
           channel URLs : https://repo.continuum.io/pkgs/free/linux-64
                          https://repo.continuum.io/pkgs/free/noarch
                          https://repo.continuum.io/pkgs/r/linux-64
                          https://repo.continuum.io/pkgs/r/noarch
                          https://repo.continuum.io/pkgs/pro/linux-64
                          https://repo.continuum.io/pkgs/pro/noarch
            config file : None
           offline mode : False
             user-agent : conda/4.3.11 requests/2.12.4 CPython/3.6.0 Linux/4.4.0-64-generic debian/stretch/sid glibc/2.23
                UID:GID : 1000:1000

[15:48] humitos: aospy>

After this, I booted the readthedocs/build:2.0 and ran into this permission problem again:

docs@40764e8adc4f:/$ conda info
Current conda install:

               platform : linux-64
          conda version : 4.2.12
       conda is private : False
      conda-env version : 4.2.12
    conda-build version : not installed
         python version : 2.7.12.final.0
       requests version : 2.11.1
       root environment : /usr/local/miniconda  (read only)
    default environment : /usr/local/miniconda
       envs directories : /home/docs/.conda/envs
                          /usr/local/miniconda/envs
          package cache : /home/docs/.conda/envs/.pkgs
                          /usr/local/miniconda/pkgs
           channel URLs : https://repo.continuum.io/pkgs/free/linux-64
                          https://repo.continuum.io/pkgs/free/noarch
                          https://repo.continuum.io/pkgs/pro/linux-64
                          https://repo.continuum.io/pkgs/pro/noarch
            config file : None
           offline mode : False

docs@40764e8adc4f:/$ git clone https://github.com/spencerahill/aospy.git
fatal: could not create work tree dir 'aospy': Permission denied
docs@40764e8adc4f:/$ cd
docs@40764e8adc4f:~$ git clone https://github.com/spencerahill/aospy.git
Cloning into 'aospy'...
remote: Counting objects: 2156, done.
remote: Compressing objects: 100% (22/22), done.
remote: Total 2156 (delta 7), reused 1 (delta 1), pack-reused 2133
Receiving objects: 100% (2156/2156), 3.62 MiB | 502.00 KiB/s, done.
Resolving deltas: 100% (1468/1468), done.
Checking connectivity... done.
docs@40764e8adc4f:~$ cd aospy/
docs@40764e8adc4f:~/aospy$ conda env create --name latest --file docs/environment.yml 
Fetching package metadata .........
Solving package specifications: ..........
Fetching packages ...
ca-certificate 100% |##############################################################################################################################################| Time: 0:00:00 240.00 kB/s
[ ... snipped ... ]

xz-5.2.2-0.tar 100% |##############################################################################################################################################| Time: 0:00:02 356.06 kB/s
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
An unexpected error has occurred.
Please consider posting the following information to the
conda GitHub issue tracker at:

    https://github.com/conda/conda/issues



Current conda install:

               platform : linux-64
          conda version : 4.2.12
       conda is private : False
      conda-env version : 4.2.12
    conda-build version : not installed
         python version : 2.7.12.final.0
       requests version : 2.11.1
       root environment : /usr/local/miniconda  (read only)
    default environment : /usr/local/miniconda
       envs directories : /home/docs/.conda/envs
                          /usr/local/miniconda/envs
          package cache : /home/docs/.conda/envs/.pkgs
                          /usr/local/miniconda/pkgs
           channel URLs : https://repo.continuum.io/pkgs/free/linux-64
                          https://repo.continuum.io/pkgs/free/noarch
                          https://repo.continuum.io/pkgs/pro/linux-64
                          https://repo.continuum.io/pkgs/pro/noarch
            config file : None
           offline mode : False



`$ /usr/local/miniconda/bin/conda-env create --name latest --file docs/environment.yml`




    Traceback (most recent call last):
      File "/usr/local/miniconda/lib/python2.7/site-packages/conda/exceptions.py", line 479, in conda_exception_handler
        return_value = func(*args, **kwargs)
      File "/usr/local/miniconda/lib/python2.7/site-packages/conda_env/cli/main_create.py", line 111, in execute
        installer.install(prefix, pkg_specs, args, env)
      File "/usr/local/miniconda/lib/python2.7/site-packages/conda_env/installers/conda.py", line 38, in install
        raise CondaRuntimeError('RuntimeError: %s' % e)
    CondaRuntimeError: Runtime error: RuntimeError: Runtime error: Could not open u'/usr/local/miniconda/pkgs/yaml-0.1.6-0.tar.bz2.part' for writing ([Errno 13] Permission denied: u'/usr/local/miniconda/pkgs/yaml-0.1.6-0.tar.bz2.part').

docs@40764e8adc4f:~/aospy$ ls -la /usr/local/miniconda/pkgs/
total 112
drwxr-xr-x 20 root root  4096 Feb  5 07:43 .
drwxr-xr-x 11 root root  4096 Feb  5 07:43 ..
-rw-rw-r--  1 root root  2097 Nov  3 19:04 .cio-config.py
-rw-rw-r--  1 root root 17834 Nov  3 19:04 .install.py
drwxr-xr-x  2 root root  4096 Feb  5 07:43 cache
drwxr-xr-x  6 root root  4096 Feb  5 07:43 conda-4.2.12-py27_0
drwxr-xr-x  3 root root  4096 Feb  5 07:43 conda-env-2.6.0-0
drwxr-xr-x  4 root root  4096 Feb  5 07:43 enum34-1.1.6-py27_0
drwxr-xr-x  7 root root  4096 Feb  5 07:43 openssl-1.0.2j-0
drwxr-xr-x  5 root root  4096 Feb  5 07:43 pip-8.1.2-py27_0
drwxr-xr-x  4 root root  4096 Feb  5 07:43 pycosat-0.6.1-py27_1
drwxr-xr-x  4 root root  4096 Feb  5 07:43 pycrypto-2.6.1-py27_4
drwxr-xr-x  7 root root  4096 Feb  5 07:43 python-2.7.12-1
drwxr-xr-x  6 root root  4096 Feb  5 07:43 readline-6.2-2
drwxr-xr-x  4 root root  4096 Feb  5 07:43 requests-2.11.1-py27_0
drwxr-xr-x  4 root root  4096 Feb  5 07:43 ruamel_yaml-0.11.14-py27_0
drwxr-xr-x  5 root root  4096 Feb  5 07:43 setuptools-27.2.0-py27_0
drwxr-xr-x  6 root root  4096 Feb  5 07:43 sqlite-3.13.0-0
drwxr-xr-x  6 root root  4096 Feb  5 07:43 tk-8.5.18-0
-rw-rw-r--  1 root root  1777 Nov  3 19:04 urls
-rw-rw-r--  1 root root  1216 Nov  3 19:04 urls.txt
drwxr-xr-x  5 root root  4096 Feb  5 07:43 wheel-0.29.0-py27_0
drwxr-xr-x  5 root root  4096 Feb  5 07:43 yaml-0.1.6-0
drwxr-xr-x  5 root root  4096 Feb  5 07:43 zlib-1.2.8-3

docs@40764e8adc4f:~/aospy$ ls -la /home/docs/.conda/envs/.pkgs
total 46044
drwxr-xr-x 3 docs docs     4096 Mar  7 20:55 .
drwxr-xr-x 4 docs docs     4096 Mar  7 20:53 ..
-rw-r--r-- 1 docs docs   169592 Mar  7 20:53 ca-certificates-2017.1.23-0.tar.bz2
drwxr-xr-x 2 docs docs     4096 Mar  7 20:53 cache
-rw-r--r-- 1 docs docs  5892278 Mar  7 20:54 gettext-0.19.7-1.tar.bz2
-rw-r--r-- 1 docs docs   120285 Mar  7 20:54 graphite2-1.3.9-0.tar.bz2
-rw-r--r-- 1 docs docs 23833004 Mar  7 20:54 icu-58.1-1.tar.bz2
-rw-r--r-- 1 docs docs   939817 Mar  7 20:55 jpeg-9b-0.tar.bz2
-rw-r--r-- 1 docs docs    47736 Mar  7 20:55 libffi-3.2.1-3.tar.bz2
-rw-r--r-- 1 docs docs   287481 Mar  7 20:55 libgfortran-3.0.0-1.tar.bz2
-rw-r--r-- 1 docs docs  2077443 Mar  7 20:55 libiconv-1.14-4.tar.bz2
-rw-r--r-- 1 docs docs  1121442 Mar  7 20:55 ncurses-5.9-10.tar.bz2
-rw-r--r-- 1 docs docs   742096 Mar  7 20:55 pcre-8.39-0.tar.bz2
-rw-r--r-- 1 docs docs  3799933 Mar  7 20:55 pixman-0.34.0-0.tar.bz2
-rw-r--r-- 1 docs docs  5160488 Mar  7 20:55 sqlite-3.13.0-1.tar.bz2
-rw-r--r-- 1 docs docs  2038478 Mar  7 20:55 tk-8.5.19-1.tar.bz2
-rw-r--r-- 1 docs docs      998 Mar  7 20:55 urls.txt
-rw-r--r-- 1 docs docs   873657 Mar  7 20:55 xz-5.2.2-0.tar.bz2
docs@40764e8adc4f:~/aospy$

So, the issue is that we have two directories set as package cache, one that is accessible by the docs user and another one that only root has permission. For some reason, some packages goes to the /home/docs/.conda/envs/.pkgs (where the docs user has permission) and for some reason there are packages that goes to /usr/local/miniconda/pkgs and it fails because we are executing conda as docs user.

Another test was trying to update the conda itself inside the docker image and it also says we don't have permissions to do it:

docs@40764e8adc4f:~$ conda info
Current conda install:

               platform : linux-64
          conda version : 4.2.12
       conda is private : False
      conda-env version : 4.2.12
    conda-build version : not installed
         python version : 2.7.12.final.0
       requests version : 2.11.1
       root environment : /usr/local/miniconda  (read only)
    default environment : /usr/local/miniconda
       envs directories : /home/docs/.conda/envs
                          /usr/local/miniconda/envs
          package cache : /home/docs/.conda/envs/.pkgs
                          /usr/local/miniconda/pkgs
           channel URLs : https://repo.continuum.io/pkgs/free/linux-64
                          https://repo.continuum.io/pkgs/free/noarch
                          https://repo.continuum.io/pkgs/pro/linux-64
                          https://repo.continuum.io/pkgs/pro/noarch
            config file : None
           offline mode : False

docs@40764e8adc4f:~$ conda update conda
Fetching package metadata .......
Solving package specifications: ..........

Package plan for installation in environment /usr/local/miniconda:

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    libffi-3.2.1               |                1          38 KB
    idna-2.2                   |           py27_0         122 KB
    ipaddress-1.0.18           |           py27_0          31 KB
    pyasn1-0.1.9               |           py27_0          54 KB
    pycparser-2.17             |           py27_0         153 KB
    requests-2.13.0            |           py27_0         776 KB
    six-1.10.0                 |           py27_0          16 KB
    cffi-1.9.1                 |           py27_0         325 KB
    cryptography-1.7.1         |           py27_0         848 KB
    pyopenssl-16.2.0           |           py27_0          68 KB
    conda-4.3.14               |           py27_0         481 KB
    ------------------------------------------------------------
                                           Total:         2.8 MB

The following NEW packages will be INSTALLED:

    cffi:         1.9.1-py27_0  (soft-link)
    cryptography: 1.7.1-py27_0  (soft-link)
    idna:         2.2-py27_0    (soft-link)
    ipaddress:    1.0.18-py27_0 (soft-link)
    libffi:       3.2.1-1       (soft-link)
    pyasn1:       0.1.9-py27_0  (soft-link)
    pycparser:    2.17-py27_0   (soft-link)
    pyopenssl:    16.2.0-py27_0 (soft-link)
    six:          1.10.0-py27_0 (soft-link)

The following packages will be UPDATED:

    conda:        4.2.12-py27_0 --> 4.3.14-py27_0 (soft-link)
    requests:     2.11.1-py27_0 --> 2.13.0-py27_0 (soft-link)



CondaIOError: IO error: Missing write permissions in: /usr/local/miniconda
#
# You don't appear to have the necessary permissions to update packages
# into the install area '/usr/local/miniconda'.
# However you can clone this environment into your home directory and
# then make changes to it.
# This may be done using the command:
#
# $ conda create -n my_root --clone=/usr/local/miniconda



docs@40764e8adc4f:~$

Following the suggestion from the previous command is not a good idea 😁

docs@40764e8adc4f:~$ conda create -n my_root --clone=/usr/local/miniconda
Source:      /usr/local/miniconda
Destination: /home/docs/.conda/envs/my_root
The following packages cannot be cloned out of the root environment:
 - conda-4.2.12-py27_0
Packages: 16
Files: 0
Linking packages ...
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
[tk                  ]|########################################      WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]             |  18%
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
[pycosat             ]|##########################################################################################                         WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
[pip            WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]#############################################                     |  87%
[      COMPLETE      ]|#################################################################################################################################################################| 100%
#
# To activate this environment, use:
# > source activate my_root
#
# To deactivate this environment, use:
# > source deactivate my_root
#

docs@40764e8adc4f:~$

I installed miniconda with the docs user and it was successful:

docs@40764e8adc4f:~$ curl -O https://repo.continuum.io/miniconda/Miniconda2-latest-Linux-x86_64.sh
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 27.4M  100 27.4M    0     0   162k      0  0:02:53  0:02:53 --:--:--  141k
docs@40764e8adc4f:~$ bash Miniconda2-latest-Linux-x86_64.sh -b -p /home/docs/miniconda2
PREFIX=/home/docs/miniconda2
installing: python-2.7.13-0 ...
installing: cffi-1.9.1-py27_0 ...

[ ... snipped ... ]

docs@40764e8adc4f:~$ /home/docs/miniconda2/bin/conda info
Current conda install:

               platform : linux-64
          conda version : 4.3.11
       conda is private : False
      conda-env version : 4.3.11
    conda-build version : not installed
         python version : 2.7.13.final.0
       requests version : 2.12.4
       root environment : /home/docs/miniconda2  (writable)
    default environment : /home/docs/miniconda2
       envs directories : /home/docs/miniconda2/envs
                          /home/docs/.conda/envs
          package cache : /home/docs/miniconda2/pkgs
                          /home/docs/.conda/pkgs
           channel URLs : https://repo.continuum.io/pkgs/free/linux-64
                          https://repo.continuum.io/pkgs/free/noarch
                          https://repo.continuum.io/pkgs/r/linux-64
                          https://repo.continuum.io/pkgs/r/noarch
                          https://repo.continuum.io/pkgs/pro/linux-64
                          https://repo.continuum.io/pkgs/pro/noarch
            config file : None
           offline mode : False
             user-agent : conda/4.3.11 requests/2.12.4 CPython/2.7.13 Linux/4.4.0-64-generic debian/stretch/sid glibc/2.23
                UID:GID : 1005:205

docs@40764e8adc4f:~$ /home/docs/miniconda2/bin/conda env create --name latest --file aospy/docs/environment.yml 
Fetching package metadata ...........
Solving package specifications: .

[ ... snipped ... ]

python-3.5.3-1 100% |##############################################################################################################################################| Time: 0:01:38 196.49 kB/s

[ ... snipped ... ]

matplotlib-1.5 100% |##############################################################################################################################################| Time: 0:00:16 477.15 kB/s
#
# To activate this environment, use:
# > source activate latest
#
# To deactivate this environment, use:
# > source deactivate latest
#


docs@40764e8adc4f:~$

So, my suggestion is to install miniconda after the docs user is created (and switched to) in the docker image as in this PR: readthedocs/readthedocs-docker-images#25

Plus, if we really want to pin a version as in readthedocs/readthedocs-docker-images#23 we need to use a setting to avoid the automatic upgrade (conda config --set auto_update_conda False -- https://conda.io/docs/config.html#update-conda-automatically-auto-update-conda) but I'm not sure if that is something good to do.

ericholscher · 2017-03-08T00:06:25Z

I've tried downgrading conda, and now we get a YAML error from the conda env command. Hopefully the user permission fix will work, and we can hopefully get this to a working state.

ericholscher · 2017-03-08T00:08:13Z

It appears to be this issue: conda/conda#3738

How do we install a version of conda that isn't listed over at https://repo.continuum.io/miniconda/? Do we have to install a lower version of conda, and then use conda upgrade conda==X?

ericholscher · 2017-03-08T00:15:01Z

I've got this failing locally, and will patch it and deploy a fix in the next couple hours, hopefully.

david4096 · 2017-03-08T01:40:54Z

My latest https://readthedocs.org/projects/schemas/builds/5118842/

Traceback (most recent call last):
  File "/usr/local/miniconda/bin/conda-env", line 4, in <module>
    import conda_env.cli.main
  File "/usr/local/miniconda/lib/python2.7/site-packages/conda_env/cli/main.py", line 35, in <module>
    from . import main_attach
  File "/usr/local/miniconda/lib/python2.7/site-packages/conda_env/cli/main_attach.py", line 4, in <module>
    from ..env import from_environment
  File "/usr/local/miniconda/lib/python2.7/site-packages/conda_env/env.py", line 17, in <module>
    from . import yaml
  File "/usr/local/miniconda/lib/python2.7/site-packages/conda_env/yaml.py", line 8, in <module>
    import yaml
ImportError: No module named yaml

ericholscher · 2017-03-08T01:57:25Z

I believe this should finally be fixed. Knock on wood.

david4096 · 2017-03-08T02:03:09Z

Thanks, looking good so far! https://readthedocs.org/projects/schemas/builds/5118911/

david4096 · 2017-03-08T02:07:18Z

Great success, thanks!

spencerahill · 2017-03-08T02:11:19Z

Yes, success for us: https://readthedocs.org/projects/aospy/builds/5118901/

Thank you very much!

agjohnson · 2017-03-08T02:27:09Z

Glad to hear this is working for folks. I'll close this for now, feel free to reopen here if it isn't resolved.

humitos · 2017-03-08T03:20:34Z

I think it starts failing recently because a new version of yaml which was installed at root level, so when it tries to upgrade that package with de docs users it fails (check my log and when I said that I don't know why some packages goes to the root cache and some others to the docs cache). It's because of that...

astrojuanlu · 2017-03-08T09:11:08Z

Congratulations to the RTD devs, this wasn't an easy one :)

swainn · 2017-03-08T18:45:51Z

This is fixed for me:
https://readthedocs.org/projects/tethys-platform/builds/5122358/
I completely understand the challenges with community projects. Thank you for your time.

fmaussion mentioned this issue Feb 21, 2017

Current doc builds are broken pydata/xarray#1280

Closed

swainn mentioned this issue Mar 1, 2017

Docs not Building Due to Conda Issue tethysplatform/tethys#232

Closed

spencerahill mentioned this issue Mar 3, 2017

Make python versions supported (2.7, 3.4, 3.5, and 3.6) consistent across CI, setup.py, and conda releases spencerahill/aospy#124

Closed

dcolligan mentioned this issue Mar 3, 2017

Read the Docs (RTD) build failing ga4gh/ga4gh-server#1599

Closed

cdeil mentioned this issue Mar 5, 2017

Broken conda install for docs build on Readthedocs gammapy/gammapy#934

Closed

spencerahill mentioned this issue Mar 7, 2017

v0.1.2 announcement emails spencerahill/aospy#127

Closed

7 tasks

agjohnson mentioned this issue Mar 7, 2017

Fix conda to a particular release readthedocs/readthedocs-docker-images#22

Closed

humitos added Bug A bug Operations Operations or server issue labels Mar 7, 2017

arbennett mentioned this issue Mar 7, 2017

Feature/docs UW-Hydro/MetSim#57

Closed

2 tasks

This was referenced Mar 7, 2017

Fix conda permissions in docker image readthedocs/readthedocs-docker-images#25

Merged

Conda environment is not being built #2700

Closed

ericholscher mentioned this issue Mar 8, 2017

Conda build of environment.yml broke without changing yml file #2704

Closed

agjohnson closed this as completed Mar 8, 2017

fmaussion mentioned this issue Mar 16, 2017

"ImportError: libgeos-3.4.2.so" on read the docs conda-forge/fiona-feedstock#21

Closed

orbeckst mentioned this issue Apr 29, 2019

RTD build fails Becksteinlab/GromacsWrapper#170

Closed

swaldhoer mentioned this issue Feb 2, 2020

Update doxygen to 1.8.16 #6483

Closed

Build Fails to Create Conda Environment #2651

Build Fails to Create Conda Environment #2651

Comments

swainn commented Feb 18, 2017

Details

Expected Result

Actual Result

Things I have tried:

fmaussion commented Feb 21, 2017

swainn commented Feb 23, 2017

fmaussion commented Feb 23, 2017

fmaussion commented Feb 23, 2017

woodmd commented Feb 24, 2017

fmaussion commented Feb 28, 2017 • edited Loading

swainn commented Feb 28, 2017

spencerahill commented Mar 2, 2017

snowman2 commented Mar 3, 2017

dcolligan commented Mar 3, 2017

david4096 commented Mar 7, 2017

humitos commented Mar 7, 2017

ericholscher commented Mar 7, 2017

ericholscher commented Mar 7, 2017

snowman2 commented Mar 7, 2017

ericholscher commented Mar 7, 2017

ericholscher commented Mar 7, 2017 • edited Loading

ericholscher commented Mar 7, 2017

humitos commented Mar 7, 2017

ericholscher commented Mar 7, 2017

ericholscher commented Mar 7, 2017

snowman2 commented Mar 7, 2017

agjohnson commented Mar 7, 2017

snowman2 commented Mar 7, 2017

humitos commented Mar 7, 2017

ericholscher commented Mar 7, 2017

humitos commented Mar 7, 2017

agjohnson commented Mar 7, 2017

ericholscher commented Mar 7, 2017

agjohnson commented Mar 7, 2017

humitos commented Mar 7, 2017 • edited Loading

ericholscher commented Mar 8, 2017

ericholscher commented Mar 8, 2017 • edited Loading

ericholscher commented Mar 8, 2017

david4096 commented Mar 8, 2017

ericholscher commented Mar 8, 2017

david4096 commented Mar 8, 2017

david4096 commented Mar 8, 2017

spencerahill commented Mar 8, 2017

agjohnson commented Mar 8, 2017

humitos commented Mar 8, 2017

astrojuanlu commented Mar 8, 2017 • edited Loading

swainn commented Mar 8, 2017

fmaussion commented Feb 28, 2017 •

edited

Loading

ericholscher commented Mar 7, 2017 •

edited

Loading

humitos commented Mar 7, 2017 •

edited

Loading

ericholscher commented Mar 8, 2017 •

edited

Loading

astrojuanlu commented Mar 8, 2017 •

edited

Loading