Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade nixpkgs #3155

Merged
merged 2 commits into from
Jan 18, 2024
Merged

Conversation

wolfgangwalther
Copy link
Member

@wolfgangwalther wolfgangwalther commented Jan 11, 2024

I tried updating nixpkgs a few day ago. This didn't work out well, because the nixpkgs-unstable channel had some packages that we require broken. Then I wondered: Why do we even have nixpkgs-unstable in postgrest-nixpkgs-upgrade? So I changed that to use the latest stable channel - and then ran the upgrade again.

This time, I was able to make it work with a few changes here and there. One major change:

  • fuzzyset was marked as broken, so I replaced that.
  • fuzzyset requires text >=2.0.2 - which is only available in GHC 9.4+
  • Therefore, I upgraded to GHC 9.4.

That means that because of the fuzzyset dependency, we can't build with GHC 9.2 or GHC 9.0 anymore. I don't think that this is a problem... but let's wait and see what the Build-Cabal-Arm job says about that.

TODO:

  • Update stack

Resolves #3107
Supersedes #3112

Probably still blocked by #2834

@wolfgangwalther
Copy link
Member Author

No idea why the nix build succeeds locally, but fails in CI. Shouldn't this be reproducible?

@steve-chavez
Copy link
Member

It's taking a while to build this locally.. maybe the CI failure could be related to the Prepopulate Nix cache for Linux runners job? Perhaps opening another PR could confirm this?

@develop7
Copy link
Collaborator

No idea why the nix build succeeds locally, but fails in CI. Shouldn't this be reproducible?

Could the local environment leak into the build? Does it build in nix-shell --pure?

@steve-chavez
Copy link
Member

maybe the CI failure could be related to the Prepopulate Nix cache for Linux runners job? Perhaps opening another PR could confirm this?

Nope, just tried running the tests locally and failed too:

[nix-shell:~/Projects/postgrest]$ postgrest-test-spec
WARNING:  cast will be ignored because the source data type is a domain
psql:test/spec/fixtures/schema.sql:3158: WARNING:  cast will be ignored because the source data type is a domain
psql:test/spec/fixtures/schema.sql:3159: WARNING:  cast will be ignored because the target data type is a domain
psql:test/spec/fixtures/schema.sql:3160: WARNING:  cast will be ignored because the target data type is a domain
psql:test/spec/fixtures/schema.sql:3177: WARNING:  cast will be ignored because the source data type is a domain
psql:test/spec/fixtures/schema.sql:3178: WARNING:  cast will be ignored because the target data type is a domain
psql:test/spec/fixtures/schema.sql:3199: WARNING:  cast will be ignored because the source data type is a domain
psql:test/spec/fixtures/schema.sql:3200: WARNING:  cast will be ignored because the target data type is a domain
psql:test/spec/fixtures/schema.sql:3201: WARNING:  cast will be ignored because the target data type is a domain
psql:test/spec/fixtures/schema.sql:3220: WARNING:  cast will be ignored because the source data type is a domain
psql:test/spec/fixtures/schema.sql:3221: WARNING:  cast will be ignored because the target data type is a domain
psql:test/spec/fixtures/schema.sql:3222: WARNING:  cast will be ignored because the target data type is a domain
psql:test/spec/fixtures/schema.sql:3239: WARNING:  cast will be ignored because the source data type is a domain
psql:test/spec/fixtures/schema.sql:3240: WARNING:  cast will be ignored because the target data type is a domain
psql:test/spec/fixtures/schema.sql:3241: WARNING:  cast will be ignored because the target data type is a domain
postgrest-with-postgresql-16: You can connect with: psql 'postgres:///postgres?host=/run/user/1000/postgrest/postgrest-with-postgresql-16-sEF/socket' -U postgres
postgrest-with-postgresql-16: You can tail the logs with: tail -f /run/user/1000/postgrest/postgrest-with-postgresql-16-sEF/db.log
Warning: The package list for 'hackage.haskell.org' does not exist. Run 'cabal
update' to download it.
Warning: Requested index-state 2023-10-13T13:54:33Z is newer than
'hackage.haskell.org'! Falling back to older state ().
Resolving dependencies...
Error: cabal: Cannot run the test suite 'spec' because the solver did not find
a plan that included the test suites for postgrest-11.2.0. It is probably
worth trying again with test suites explicitly enabled in the configuration in
the cabal.project{.local} file. This will ask the solver to find a plan with
the test suites available. It will either fail with an explanation or find a
different plan that uses different versions of some other packages. Use the
'--dry-run' flag to see package versions and check that you are happy with the
choices.

Temporary directory kept at: /run/user/1000/postgrest/postgrest-with-postgresql-16-sEF

Same error message as CI

@steve-chavez
Copy link
Member

Only the tests though, running the executable works fine:

[nix-shell:~/Projects/postgrest]$ postgrest-with-postgresql-15 postgrest-run
psql:test/spec/fixtures/schema.sql:2819: WARNING:  cast will be ignored because the source data type is a domain
psql:test/spec/fixtures/schema.sql:3158: WARNING:  cast will be ignored because the source data type is a domain
psql:test/spec/fixtures/schema.sql:3159: WARNING:  cast will be ignored because the target data type is a domain
psql:test/spec/fixtures/schema.sql:3160: WARNING:  cast will be ignored because the target data type is a domain
psql:test/spec/fixtures/schema.sql:3177: WARNING:  cast will be ignored because the source data type is a domain
psql:test/spec/fixtures/schema.sql:3178: WARNING:  cast will be ignored because the target data type is a domain
psql:test/spec/fixtures/schema.sql:3199: WARNING:  cast will be ignored because the source data type is a domain
psql:test/spec/fixtures/schema.sql:3200: WARNING:  cast will be ignored because the target data type is a domain
psql:test/spec/fixtures/schema.sql:3201: WARNING:  cast will be ignored because the target data type is a domain
psql:test/spec/fixtures/schema.sql:3220: WARNING:  cast will be ignored because the source data type is a domain
psql:test/spec/fixtures/schema.sql:3221: WARNING:  cast will be ignored because the target data type is a domain
psql:test/spec/fixtures/schema.sql:3222: WARNING:  cast will be ignored because the target data type is a domain
psql:test/spec/fixtures/schema.sql:3239: WARNING:  cast will be ignored because the source data type is a domain
psql:test/spec/fixtures/schema.sql:3240: WARNING:  cast will be ignored because the target data type is a domain
psql:test/spec/fixtures/schema.sql:3241: WARNING:  cast will be ignored because the target data type is a domain
postgrest-with-postgresql-15: You can connect with: psql 'postgres:///postgres?host=/run/user/1000/postgrest/postgrest-with-postgresql-15-v6T/socket' -U postgres
postgrest-with-postgresql-15: You can tail the logs with: tail -f /run/user/1000/postgrest/postgrest-with-postgresql-15-v6T/db.log
12/Jan/2024:11:12:22 -0500: Starting PostgREST 11.2.0 (bac81f0)...
12/Jan/2024:11:12:22 -0500: Attempting to connect to the database...
12/Jan/2024:11:12:22 -0500: Connection successful
12/Jan/2024:11:12:22 -0500: Listening on port 3000
12/Jan/2024:11:12:22 -0500: Listening for notifications on the pgrst channel
12/Jan/2024:11:12:22 -0500: Config reloaded
12/Jan/2024:11:12:22 -0500: Schema cache loaded

@steve-chavez
Copy link
Member

steve-chavez commented Jan 12, 2024

Could the local environment leak into the build? Does it build in nix-shell --pure?

nix-shell --pure and postgrest-test-spec also fails for me.

@steve-chavez
Copy link
Member

Probably still blocked by #2834

No problem with the above from my side. Let's just upgrade and figure it out later.

@wolfgangwalther
Copy link
Member Author

wolfgangwalther commented Jan 12, 2024

I'm not sure what to conclude from the observations here. postgrest-test-spec is

  • failing in CI (nix-env -iA)
  • passing for me (nix-env -iA)
  • passing for me (nix-shell)
  • passing for me (nix-shell pure)
  • failing for Steve (nix-shell)
  • passing for Steve (nix-shell pure)

Correct?

I wonder what I need to do to make it fail.. so I can actually debug it.

@wolfgangwalther
Copy link
Member Author

I wonder what I need to do to make it fail.. so I can actually debug it.

Aha. I deleted my ~/.cabal directory. And now it's failing for me, too.

So nix-shell --pure does not protect from that.

@wolfgangwalther
Copy link
Member Author

Doing a cabal update makes it work again...

... but this is all confusing. We should not need to run cabal update - the nix-shell environment should provide all the packages we need. Clearly it doesn't, yet.

@steve-chavez
Copy link
Member

passing for Steve (nix-shell pure)
Correct?

My bad, postgrest-test-spec also fails for me with nix-shell --pure. (somehow switched branches when trying that)

stack.yaml Outdated Show resolved Hide resolved
@wolfgangwalther
Copy link
Member Author

Added to commits to test building with GHC 9.6.x and GHC 9.8.x just out of curiosity..

@wolfgangwalther
Copy link
Member Author

Added to commits to test building with GHC 9.6.x and GHC 9.8.x just out of curiosity..

Seems like our dependencies are not ready for GHC 9.6+, yet. Will drop those for now.

Previously this command upgraded to the latest unstable version of nixpkgs,
but this was often broken. Taking the latest stable branch should give
better results.
@develop7
Copy link
Collaborator

Couldn't help but notice "prepopulate nix cache" job is building the xgcc from scratch (https://github.com/PostgREST/postgrest/actions/runs/7534062527/job/20507765748?pr=3155#step:4:8232) taking far too much time. Any idea why?

@wolfgangwalther
Copy link
Member Author

Couldn't help but notice "prepopulate nix cache" job is building the xgcc from scratch (https://github.com/PostgREST/postgrest/actions/runs/7534062527/job/20507765748?pr=3155#step:4:8232) taking far too much time. Any idea why?

It's expected that the toolchain needs to be rebuilt after a nixpkgs upgrade, I think. I have not looked at the prepopulate nix cache job and the related changes in the last months, so I can't tell whether it does something odd there. But a complete rebuild after a nixpkgs update often takes a lot of time - and in the small machines that are run in CI, 5+ hours are expected for me.

Am I missing something?

Copy link
Member

@steve-chavez steve-chavez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@develop7
Copy link
Collaborator

develop7 commented Jan 18, 2024

and in the small machines that are run in CI, 5+ hours are expected for me

@wolfgangwalther I see. Are you suggesting these long-building outputs eventually make their way to PostgREST's Cachix? Fine by me, then.

@wolfgangwalther
Copy link
Member Author

Are you suggesting these long-building outputs eventually make their way to PostgREST's Cachix?

That's my understanding, yes. Once those jobs run on the main branch, they will actually push to cachix.

@wolfgangwalther wolfgangwalther merged commit 6682bae into PostgREST:main Jan 18, 2024
30 of 31 checks passed
@wolfgangwalther wolfgangwalther deleted the upgrade-nixpkgs branch January 18, 2024 17:54
@wolfgangwalther wolfgangwalther mentioned this pull request Feb 7, 2024
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

Support ghc 9.4
3 participants