Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

miscellaneous swig3 -> swig4 updates #337609

Closed
wants to merge 15 commits into from
Closed

Conversation

bcdarwin
Copy link
Member

Description of changes

Unpin a bunch of swig3 dependencies and re-pin them at swig4 (to avoid a treewide swig = swig4 bump yet, but willing to do that).

Everything here should be straightforward - no other non-cosmetic changes required and all packages building fine before and after the bump.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@emilazy
Copy link
Member

emilazy commented Aug 27, 2024

(These results are from before the latest batch of commits.)

Result of nixpkgs-review pr 337609 run on aarch64-linux 1

14 packages marked as broken and skipped:
  • python311Packages.n3fit
  • python311Packages.n3fit.dist
  • python311Packages.validphys2
  • python311Packages.validphys2.dist
  • python311Packages.worldengine
  • python311Packages.worldengine.dist
  • python312Packages.shimmy
  • python312Packages.shimmy.dist
  • python312Packages.validphys2
  • python312Packages.validphys2.dist
  • python312Packages.worldengine
  • python312Packages.worldengine.dist
  • validphys2
  • validphys2.dist
11 packages failed to build:
  • apacheHttpdPackages.mod_tile
  • global-platform-pro
  • python311Packages.django-bootstrap4
  • python311Packages.django-bootstrap4.dist
  • python311Packages.django-bootstrap5
  • python311Packages.django-bootstrap5.dist
  • python312Packages.pivy
  • python312Packages.pivy.dist
  • pytrainer
  • pytrainer.dist
  • t-rex
151 packages built:
  • cloudcompare
  • cvc4
  • entwine
  • fityk
  • freecad
  • freecad-wayland
  • gdal (python312Packages.gdal)
  • gdalMinimal
  • gplates
  • grass
  • mapcache
  • mapnik
  • mapproxy
  • mapproxy.dist
  • mapserver
  • merkaartor
  • mysql-workbench
  • nnpdf (python312Packages.nnpdf)
  • octavePackages.mapping
  • openorienteering-mapper
  • paraview
  • pdal
  • perl536Packages.Tirex
  • perl536Packages.Tirex.devdoc
  • perl538Packages.Tirex
  • perl538Packages.Tirex.devdoc
  • postgresql12JitPackages.postgis
  • postgresql12JitPackages.postgis.doc
  • postgresql12Packages.postgis
  • postgresql12Packages.postgis.doc
  • postgresql13JitPackages.postgis
  • postgresql13JitPackages.postgis.doc
  • postgresql13Packages.postgis
  • postgresql13Packages.postgis.doc
  • postgresql14JitPackages.postgis
  • postgresql14JitPackages.postgis.doc
  • postgresql14Packages.postgis
  • postgresql14Packages.postgis.doc
  • postgresql15JitPackages.postgis
  • postgresql15JitPackages.postgis.doc
  • postgresql15Packages.postgis
  • postgresql15Packages.postgis.doc
  • postgresql16JitPackages.postgis
  • postgresql16JitPackages.postgis.doc
  • postgresql16Packages.postgis
  • postgresql16Packages.postgis.doc
  • proot
  • python311Packages.babeltrace2
  • python311Packages.babeltrace2.dev
  • python311Packages.babeltrace2.man
  • python311Packages.bsuite
  • python311Packages.bsuite.dist
  • python311Packages.cartopy
  • python311Packages.cartopy.dist
  • python311Packages.fiona
  • python311Packages.fiona.dist
  • python311Packages.folium
  • python311Packages.folium.dist
  • python311Packages.gdal
  • python311Packages.geodatasets
  • python311Packages.geodatasets.dist
  • python311Packages.geopandas
  • python311Packages.geopandas.dist
  • python311Packages.geoparquet
  • python311Packages.geoparquet.dist
  • python311Packages.inequality
  • python311Packages.inequality.dist
  • python311Packages.libpysal
  • python311Packages.libpysal.dist
  • python311Packages.mapclassify
  • python311Packages.mapclassify.dist
  • python311Packages.momepy
  • python311Packages.momepy.dist
  • python311Packages.morecantile
  • python311Packages.morecantile.dist
  • python311Packages.nnpdf
  • python311Packages.osmnx
  • python311Packages.osmnx.dist
  • python311Packages.pivy
  • python311Packages.pivy.dist
  • python311Packages.plotnine
  • python311Packages.plotnine.dist
  • python311Packages.pyogrio
  • python311Packages.pyogrio.dist
  • python311Packages.python-mapnik
  • python311Packages.python-mapnik.dist
  • python311Packages.rasterio
  • python311Packages.rasterio.dist
  • python311Packages.rio-tiler
  • python311Packages.rio-tiler.dist
  • python311Packages.rioxarray
  • python311Packages.rioxarray.dist
  • python311Packages.shimmy
  • python311Packages.shimmy.dist
  • python311Packages.wktutils
  • python311Packages.wktutils.dist
  • python312Packages.babeltrace2
  • python312Packages.babeltrace2.dev
  • python312Packages.babeltrace2.man
  • python312Packages.bsuite
  • python312Packages.bsuite.dist
  • python312Packages.cartopy
  • python312Packages.cartopy.dist
  • python312Packages.django-bootstrap4
  • python312Packages.django-bootstrap4.dist
  • python312Packages.django-bootstrap5
  • python312Packages.django-bootstrap5.dist
  • python312Packages.fiona
  • python312Packages.fiona.dist
  • python312Packages.folium
  • python312Packages.folium.dist
  • python312Packages.geodatasets
  • python312Packages.geodatasets.dist
  • python312Packages.geopandas
  • python312Packages.geopandas.dist
  • python312Packages.geoparquet
  • python312Packages.geoparquet.dist
  • python312Packages.inequality
  • python312Packages.inequality.dist
  • python312Packages.libpysal
  • python312Packages.libpysal.dist
  • python312Packages.mapclassify
  • python312Packages.mapclassify.dist
  • python312Packages.momepy
  • python312Packages.momepy.dist
  • python312Packages.morecantile
  • python312Packages.morecantile.dist
  • python312Packages.osmnx
  • python312Packages.osmnx.dist
  • python312Packages.plotnine
  • python312Packages.plotnine.dist
  • python312Packages.pyogrio
  • python312Packages.pyogrio.dist
  • python312Packages.python-mapnik
  • python312Packages.python-mapnik.dist
  • python312Packages.rasterio
  • python312Packages.rasterio.dist
  • python312Packages.rio-tiler
  • python312Packages.rio-tiler.dist
  • python312Packages.rioxarray
  • python312Packages.rioxarray.dist
  • python312Packages.wktutils
  • python312Packages.wktutils.dist
  • qgis
  • qgis-ltr
  • qmapshack
  • saga
  • solc
  • sumo
  • vpv
  • zeek

Result of nixpkgs-review pr 337609 run on aarch64-darwin 1

19 packages marked as broken and skipped:
  • gplates
  • python311Packages.n3fit
  • python311Packages.n3fit.dist
  • python311Packages.shimmy
  • python311Packages.shimmy.dist
  • python311Packages.validphys2
  • python311Packages.validphys2.dist
  • python311Packages.worldengine
  • python311Packages.worldengine.dist
  • python312Packages.shimmy
  • python312Packages.shimmy.dist
  • python312Packages.validphys2
  • python312Packages.validphys2.dist
  • python312Packages.worldengine
  • python312Packages.worldengine.dist
  • sumo
  • validphys2
  • validphys2.dist
  • vpv
86 packages failed to build:
  • gdal (python312Packages.gdal)
  • gdalMinimal
  • grass
  • mapcache
  • mapnik
  • mapproxy
  • mapproxy.dist
  • mapserver
  • merkaartor
  • octavePackages.mapping
  • openorienteering-mapper
  • pdal
  • perl536Packages.Tirex
  • perl536Packages.Tirex.devdoc
  • perl538Packages.Tirex
  • perl538Packages.Tirex.devdoc
  • postgresql12JitPackages.postgis
  • postgresql12JitPackages.postgis.doc
  • postgresql12Packages.postgis
  • postgresql12Packages.postgis.doc
  • postgresql13JitPackages.postgis
  • postgresql13JitPackages.postgis.doc
  • postgresql13Packages.postgis
  • postgresql13Packages.postgis.doc
  • postgresql14JitPackages.postgis
  • postgresql14JitPackages.postgis.doc
  • postgresql14Packages.postgis
  • postgresql14Packages.postgis.doc
  • postgresql15JitPackages.postgis
  • postgresql15JitPackages.postgis.doc
  • postgresql15Packages.postgis
  • postgresql15Packages.postgis.doc
  • postgresql16JitPackages.postgis
  • postgresql16JitPackages.postgis.doc
  • postgresql16Packages.postgis
  • postgresql16Packages.postgis.doc
  • python311Packages.osmnx
  • python311Packages.osmnx.dist
  • python311Packages.python-mapnik
  • python311Packages.python-mapnik.dist
  • python312Packages.bsuite
  • python312Packages.bsuite.dist
  • python312Packages.cartopy
  • python312Packages.cartopy.dist
  • python312Packages.django-bootstrap4
  • python312Packages.django-bootstrap4.dist
  • python312Packages.django-bootstrap5
  • python312Packages.django-bootstrap5.dist
  • python312Packages.fiona
  • python312Packages.fiona.dist
  • python312Packages.folium
  • python312Packages.folium.dist
  • python312Packages.geodatasets
  • python312Packages.geodatasets.dist
  • python312Packages.geopandas
  • python312Packages.geopandas.dist
  • python312Packages.geoparquet
  • python312Packages.geoparquet.dist
  • python312Packages.inequality
  • python312Packages.inequality.dist
  • python312Packages.libpysal
  • python312Packages.libpysal.dist
  • python312Packages.mapclassify
  • python312Packages.mapclassify.dist
  • python312Packages.momepy
  • python312Packages.momepy.dist
  • python312Packages.morecantile
  • python312Packages.morecantile.dist
  • python312Packages.osmnx
  • python312Packages.osmnx.dist
  • python312Packages.plotnine
  • python312Packages.plotnine.dist
  • python312Packages.pyogrio
  • python312Packages.pyogrio.dist
  • python312Packages.python-mapnik
  • python312Packages.python-mapnik.dist
  • python312Packages.rasterio
  • python312Packages.rasterio.dist
  • python312Packages.rio-tiler
  • python312Packages.rio-tiler.dist
  • python312Packages.rioxarray
  • python312Packages.rioxarray.dist
  • python312Packages.wktutils
  • python312Packages.wktutils.dist
  • saga
  • t-rex
45 packages built:
  • cvc4
  • nnpdf (python312Packages.nnpdf)
  • python311Packages.bsuite
  • python311Packages.bsuite.dist
  • python311Packages.cartopy
  • python311Packages.cartopy.dist
  • python311Packages.django-bootstrap4
  • python311Packages.django-bootstrap4.dist
  • python311Packages.django-bootstrap5
  • python311Packages.django-bootstrap5.dist
  • python311Packages.fiona
  • python311Packages.fiona.dist
  • python311Packages.folium
  • python311Packages.folium.dist
  • python311Packages.gdal
  • python311Packages.geodatasets
  • python311Packages.geodatasets.dist
  • python311Packages.geopandas
  • python311Packages.geopandas.dist
  • python311Packages.geoparquet
  • python311Packages.geoparquet.dist
  • python311Packages.inequality
  • python311Packages.inequality.dist
  • python311Packages.libpysal
  • python311Packages.libpysal.dist
  • python311Packages.mapclassify
  • python311Packages.mapclassify.dist
  • python311Packages.momepy
  • python311Packages.momepy.dist
  • python311Packages.morecantile
  • python311Packages.morecantile.dist
  • python311Packages.nnpdf
  • python311Packages.plotnine
  • python311Packages.plotnine.dist
  • python311Packages.pyogrio
  • python311Packages.pyogrio.dist
  • python311Packages.rasterio
  • python311Packages.rasterio.dist
  • python311Packages.rio-tiler
  • python311Packages.rio-tiler.dist
  • python311Packages.rioxarray
  • python311Packages.rioxarray.dist
  • python311Packages.wktutils
  • python311Packages.wktutils.dist
  • zeek

Linux looks to be in a good state. Darwin is doing badly, but my (naive) regression script points to gdal as the only new failure driving a cascade of other ones, but the failure doesn’t look like something obviously SWIG‐related. I’ll run the build again and see if it’s just flaky.

Can I ask your intentions for this PR – are you going to try and get everything not on swig4 onto it one by one, or just handling some simple cases for now? I’m definitely in favour of switching the default, and I’d prefer the things pinned to swig4 here lose their version pin once that happens, to reduce the likelihood of future spurious pins. I think it would be okay to just switch swig to swig4 (presumably in the next staging cycle), pin what fails back to swig3, and then tackle that long tail, which might be easier than hunting down unnecessary pins one‐by‐one like this. But you’re the one doing the work here, so I’ll follow your lead; just let me know what your plans are :)

@bcdarwin
Copy link
Member Author

bcdarwin commented Aug 27, 2024

(These results are from before the latest batch of commits.)

Result of nixpkgs-review pr 337609 run on aarch64-linux 1
14 packages marked as broken and skipped:
11 packages failed to build:
151 packages built:

Result of nixpkgs-review pr 337609 run on aarch64-darwin 1
19 packages marked as broken and skipped:
86 packages failed to build:
45 packages built:

Linux looks to be in a good state. Darwin is doing badly, but my (naive) regression script points to gdal as the only new failure driving a cascade of other ones, but the failure doesn’t look like something obviously SWIG‐related. I’ll run the build again and see if it’s just flaky.

Can I ask your intentions for this PR – are you going to try and get everything not on swig4 onto it one by one, or just handling some simple cases for now? I’m definitely in favour of switching the default, and I’d prefer the things pinned to swig4 here lose their version pin once that happens, to reduce the likelihood of future spurious pins. I think it would be okay to just switch swig to swig4 (presumably in the next staging cycle), pin what fails back to swig3, and then tackle that long tail, which might be easier than hunting down unnecessary pins one‐by‐one like this. But you’re the one doing the work here, so I’ll follow your lead; just let me know what your plans are :)

I was thinking that approach of just bumping the default (and maybe mass-unpinning remaining swig3, but probably not there are only a few explicit swig3 cases left) might make more sense rather than (finishing the current approach and then) doing a bunch of swig4 -> swig replacements after bumping the default; happy to close this PR and do that instead.

@emilazy
Copy link
Member

emilazy commented Aug 27, 2024

I think it would be reasonable just to do the bump, if most of the users of it are expected to take the upgrade as uneventfully as these. We’re going to be doing the same for FFmpeg in the next staging cycle, so pinning stuff that breaks to swig3 will just be more of the same, especially if you’re willing to help triage :)

OTOH, this PR is clearly an improvement so if you wanted to continue with this approach that’s fine with me too. (And of course explicit unnecessary swig3 pins should definitely go.)

@bcdarwin
Copy link
Member Author

I think it would be reasonable just to do the bump, if most of the users of it are expected to take the upgrade as uneventfully as these. We’re going to be doing the same for FFmpeg in the next staging cycle, so pinning stuff that breaks to swig3 will just be more of the same, especially if you’re willing to help triage :)

OTOH, this PR is clearly an improvement so if you wanted to continue with this approach that’s fine with me too. (And of course explicit unnecessary swig3 pins should definitely go.)

Sounds good to me - I'll close this PR and create one PR to lift all possible explicit uses of swig3 and another onto staging to bump the default.

@emilazy
Copy link
Member

emilazy commented Aug 27, 2024

There’s actually fewer mentions of SWIG in‐tree than I thought, so we can probably just get everything built and fix any regressions before merge, even. Happy to help dig up the regressions if you open a PR to bump the version.

@bcdarwin bcdarwin closed this Aug 27, 2024
@bcdarwin
Copy link
Member Author

#337624

@bcdarwin bcdarwin deleted the swig-updates branch August 28, 2024 00:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants