Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

After update to 5.0 cached dependencies are built/downloaded again #27

Closed
develop7 opened this issue Jan 26, 2024 · 11 comments
Closed

After update to 5.0 cached dependencies are built/downloaded again #27

develop7 opened this issue Jan 26, 2024 · 11 comments

Comments

@develop7
Copy link

Before the upgrade to 5.0 nix-env -iA tests was a no-op because of the cache previously created in https://github.com/PostgREST/postgrest/actions/runs/7612462322/job/20730136624, after the upgrade it isn't anymore — https://github.com/PostgREST/postgrest/actions/runs/7614272320/job/20736483790#step:3:112.

Note that tar error messages are gone too. Does it mean the cache is unpacked somewhere else?

develop7 added a commit to develop7/postgrest that referenced this issue Jan 26, 2024
Upgrading c-n-action seems to break restoring Nix store rendering the cache useless (nix-community/cache-nix-action#27). This rolls the change back.
wolfgangwalther pushed a commit to PostgREST/postgrest that referenced this issue Jan 26, 2024
Upgrading c-n-action seems to break restoring Nix store rendering the cache useless (nix-community/cache-nix-action#27). This rolls the change back.
@deemp
Copy link
Collaborator

deemp commented Feb 13, 2024

@develop7, sorry for a late response.

Note that tar error messages are gone too.

I asked tar to --skip-old-files. So, it doesn't try to overwrite existing files and doesn't print errors. I wonder if this approach swallows some important errors.

https://github.com/PostgREST/postgrest/actions/runs/7614272320/job/20736483790#step:3:78

Does it mean the cache is unpacked somewhere else?

As I've written in the README, please, try to run https://github.com/mxschmitt/action-tmate right after the cache-nix-action step and inspect the Nix store.

@deemp
Copy link
Collaborator

deemp commented Feb 15, 2024

@develop7, I'll appreciate if you provide the steps to reproduce this behavior in a fork of postgrest. I mean, what actions should run and in what order. Preferrably, a minimal subset of actions.

@deemp
Copy link
Collaborator

deemp commented Mar 10, 2024

Okay, I see this problem in workflows here.

Apparently, saving a cache works incorrectly.

#879 builds two derivations and saves a cache.
https://github.com/nix-community/cache-nix-action/actions/runs/8220989301/job/22480792618#step:6:13

 these 2 derivations will be built:
  /nix/store/iil2zlkjadz68ihx4ccl85rv6798idkq-install.drv
  /nix/store/hxsbw2mdgzawjb32r1xn58i4pw5ggvzv-install.drv

#880 restores the cache and builds the same derivations
https://github.com/nix-community/cache-nix-action/actions/runs/8221051128/job/22480932943#step:6:13

these 2 derivations will be built:
  /nix/store/iil2zlkjadz68ihx4ccl85rv6798idkq-install.drv
  /nix/store/hxsbw2mdgzawjb32r1xn58i4pw5ggvzv-install.drv

@deemp
Copy link
Collaborator

deemp commented Mar 10, 2024

I re-ran one of the workflows.

https://github.com/nix-community/cache-nix-action/actions/runs/8222779566

In the build job, I restored the cache, installed nix 2.19.3 and connected to the runner via tmate.

I checked the current nix store.

ls /nix/store/ | grep '\-install$'
5qd02ybpcmqmr6vjz2c4bb8ivs8iz8yi-install
lw3yhss8q2fd6fhpc9c58357ba12yhm3-install

I saved the list of nix store paths to a file and sorted it.

ls /nix/store > a.cur
sort a.cur -o a.cur

Next, I ran the install script.

nix run .#install

Nix downloaded nixpkgs and other paths for some reason.

I saved the new list of nix store paths to a file and sorted it.

ls /nix/store > a.new
sort a.new -o a.new

I took a diff of two files and got no changes

diff a.cur a.new

I checked the results of running the write package.
Again, the store didn't change.

So, all necessary paths were in the store at the time of running the packages.
However, they weren't used.

@deemp
Copy link
Collaborator

deemp commented Mar 10, 2024

Note that tar error messages are gone too.

I suspect nix keeps info about the store in certain files, and these files are in the cache.
However due to --skip-old-files, these files don't overwrite files that exist after a fresh Nix installation.
Hence, Nix doesn't know about some paths coming from the cache.

I don't know how to "repair" the store, though.

So, for now, I'll remove --skip-old-files and hide the restore logs, as you suggested (link).

@deemp
Copy link
Collaborator

deemp commented Mar 10, 2024

In fact, I decided to ignore the existing /nix/store paths when restoring. Now, there are almost no warnings and cache restoration is slightly faster.

@deemp
Copy link
Collaborator

deemp commented Apr 5, 2024

@develop7, does your problem persist?

@develop7
Copy link
Author

Hey @deemp thank you for looking into the issue; we've phased c-n-a out for our CI setup because caching nix store was actually slower than downloading artifacts from cache.nix.org.

@deemp
Copy link
Collaborator

deemp commented May 15, 2024

Okay, I'm closing this issue then.

@deemp deemp closed this as completed May 15, 2024
@deemp
Copy link
Collaborator

deemp commented May 15, 2024

Indeed, the run times with and without caching are similar.

image

@develop7
Copy link
Author

Indeed, the run times with and without caching are similar.

I mean, they could be better on real hardware or with more generous CPU and I/O quota for GA workers but oh well

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants