Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Caching conda environment: Cannot utime: Operation not permitted #266

Open
kmiller96 opened this issue Dec 6, 2022 · 1 comment
Open

Comments

@kmiller96
Copy link

Hi! 👋 first of all, thanks for creating this action! I use it lots of my work.

I've been looking into speeding up my CI/CD pipelines and am currently investigating using your recommendation to cache the environments to achieve this.

My pipeline has no issues when it goes to save the cache. It also has no issues in finding the cached environments. However, the deserialisation of the cache fails due to some weird permissions issues. The two key lines are:

/usr/bin/tar: ../../../../../usr/share/miniconda/envs: Cannot utime: Operation not permitted
/usr/bin/tar: ../../../../../usr/share/miniconda/envs: Cannot change mode to rwxr-xr-x: Operation not permitted
/usr/bin/tar: Exiting with failure status due to previous errors

There is an unanswered stack exchange question posing this same question so I know it's not an isolated issue.

Here is what my configuration looks like:

jobs:
  example:
    runs-on: ubuntu-latest
    
    ...

    jobs:
      - name: Checkout Repo
        uses: actions/checkout@v3

      ...

      - name: Setup Mamba
        uses: conda-incubator/setup-miniconda@v2
        with:
          python-version: 3.9
          channels: conda-forge
          channel-priority: flexible

          mamba-version: "*"
          use-mamba: true

          auto-activate-base: false
          activate-environment: myenv

      - name: Set environment variables used in cache
        run: |
          echo "DATE=$(date +'%Y%m%d')" >> $GITHUB_ENV

      - name: Cache Mamba
        id: cache
        uses: actions/cache@v3
        env:
          CACHE_NUMBER: 0 # Increment this number if you wish to hard-refresh the cache.
        with:
          path: /usr/share/miniconda/envs
          key: conda-${{ env.DATE }}-${{ env.CACHE_NUMBER }}

      - name: Install development dependencies (if invalid cache)
        run: conda env update -n myenv -f environment.yaml
        if: steps.cache.outputs.cache-hit != 'true'

      ...

I have hard coded the cache paths as I couldn't quite get the templated ${{ env.CONDA }} to work for some reason, but I'm going to tackle that problem separately from this issue.

Any advice on how to solve this problem? Any and all help is appreciated!

@msamo-n
Copy link

msamo-n commented Feb 15, 2023

Hi.
My solution is not to cache whole /usr/share/miniconda/envs, but only /usr/share/miniconda/envs/my-shiny-env. Seems working.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants