CI failing for mysterious reasons

[pinin4fjords]

Hi all,

I've been having trouble recently making trivial tweaks to recipes, for example see #11292.

Basically the CI fails. When I check the outputs in Circle CI the package downloads and installs, and the tests all pass, but there's a copy error like in this PR:

"cp: cannot stat `/opt/conda/conda-bld/noarch/r-seurat-scripts-0.0.2-0.tar.bz2': No such file or directory"

This seems to be somewhat intermittent, seemed to go away but now it's recurred.

I'm afraid I'm not yet familiar with the nuts and bolts here to provide diagnosis or cure- could anyone else suggest a solution?

Many thanks,

Jon

[dpryan79]

You may need to chmod a+ws something/conda-bld/noarch and linux-64 as well. When things are run in the docker container they're done as a different user and then written to that directory (and a chown run on them). If the user that runs things in the docker container doesn't have write permissions to the aforementioned directories then you'll get an error like that.

[pinin4fjords]

Thanks @dpryan79. Where should that chmod'ing be done?

I couldn't get the container-based testing up and running on my local machine (I just use the containerless tests), so I'm only talking about the testing associated with the above PR to this repo.

[dpryan79]

You'd need to chmod where ever you've installed conda.

[pcm32]

Thanks @dpryan79 we are a bit lost still though. The explanation makes sense (privileges issue), but from the two files that we commit in the bioconda recipe (build.sh and meta.yml), we fail to see where we would enact such changes. Does the build.sh should include such logic, so that after inferring where things will be built, it makes a chmod on those folders?

[pcm32]

The error we are getting is on the CI itself, not locally: https://circleci.com/gh/bioconda/bioconda-recipes/29369#tests/containers/0

[dpryan79]

Try getting rid of noarch.

[pinin4fjords]

Okay, tried that, same issue. Very puzzling.

[pinin4fjords]

@bgruening - you were kind enough to help with these recipes to begin with. Can you by any chance identify the that's causing the CI to fail in #11292?

[chapmanb]

Jonathan;
I ran into the same issue with an update and did some experiments. I don't understand why but it turns out adding r-base to host fixes the issue:

requirements:
  host:
     - r-base

in addition to having it in run. #11394 is the PR I debugged on if you want to follow along with all my attempts. Hope adding that to your meta.yaml will get your recipe built.

[pinin4fjords]

Thanks @chapmanb - I tried it, but no joy.

[pinin4fjords]

Happening again in #11510 :-(

[chapmanb]

This one looks like it also doesn't have r-base under the host section, only under run. I know you mentioned you tested this, but I couldn't find the test commit you did in #11292 to debug further from my side. Adding this to #11510 might hopefully resolve the issue.

[pinin4fjords]

@chapmanb I just reset the git on #11292. But for completeness I've made the change to #11510 doesn't solve the problem.

[chapmanb]

Thanks for confirming. Sorry, that is all the magic I have and resolved my issue but glad you're pursuing R pinning with Bjorn as that has solved other issues with me in the past. Fingers crossed this combo will get you through this and thanks for all the work trying to debug this.

[pinin4fjords]

Thanks @chapmanb. It seems to be that in my case pinning the R version to match the pinned Seurat version (with host entry as you suggested) is what sorted things out.