netbox: 3.3.9 -> 3.4.7, netbox_3_3: init at 3.3.10, RFC42-style options, more tests #206983

minijackson · 2022-12-20T09:36:34Z

Description of changes

3.4.0 release notes (with breaking changes)
3.4.1 release notes
3.4.2 release notes
3.4.3 release notes
3.4.4 release notes
3.4.5 release notes

Also added RFC42-style options, which introduces a new pythonVars format which might be interesting for other Django apps.

Also extended the test, since we missed the Django 4.1 incompatibility / LDAP integration issue earlier.

Added myself as maintainer, since I will be probably using it for quite some time ^^

Things done

@RaitoBezarius @n0emis

RaitoBezarius · 2022-12-21T23:46:50Z

Thank you for this amazing change, I wanted to revisit our options since a long time, thank you for doing it. I hope to take a look to this in the next week.

centromere · 2023-01-20T20:00:30Z

3.4.3 has been released.

@RaitoBezarius, are you able to give this a review?

minijackson · 2023-01-21T15:09:28Z

Upgraded to 3.4.3, and rebased due to release note conflicts

RaitoBezarius

Well, all of this is very high quality. :)
My only worries are the following:

should not we introduce a state version mechanism?
should we introduce a flag to disable automatic migrations? (to prevent any dataloss)

RaitoBezarius · 2023-01-25T16:29:14Z

pkgs/servers/web-apps/netbox/default.nix


    format = "other";

    src = fetchFromGitHub {
      owner = "netbox-community";
      repo = pname;
      rev = "refs/tags/v${version}";
-      sha256 = "sha256-KhnxD5pjlEIgISl4RMbhLCDwgUDfGFRi88ZcP1ndMhI=";
+      sha256 = "sha256-M/CbnlnRaWuwcJJofY5mPpVTl9zYrApsXqV9hilNyvw=";
    };

    patches = [
      # Allow setting the STATIC_ROOT from within the configuration and setting a custom redis URL
      ./config.patch


Did we try to upstream this?

Not yet, but I think we should

pkgs/servers/web-apps/netbox/default.nix

RaitoBezarius · 2023-03-11T21:08:55Z

@minijackson Can I do something to help you on this PR? (also, can we bump it to the latest version?)

minijackson · 2023-03-12T14:54:57Z

@minijackson Can I do something to help you on this PR? (also, can we bump it to the latest version?)

On it! I just had to have some time to do it.

RaitoBezarius

The only problem I "still have" regarding this PR is the automatic migrations.
But it is hard to take a decision.
Well, we have explicit release notes about it, so this is only a weak blocker for me.
We should definitely ask what is upstream opinion on this now and if we need to introduce proper stateVersion and graceful upgrades.

RaitoBezarius · 2023-03-12T16:34:30Z

Result of nixpkgs-review pr 206983 run on x86_64-linux 1

2 packages blacklisted:

nixos-install-tools
tests.nixos-functions.nixos-test

1 package failed to build:

netbox

RaitoBezarius · 2023-03-12T16:35:42Z

Redis is failing to build for me:

> !!! WARNING The following tests failed:
       >
       > *** [err]: AOF will open a temporary INCR AOF to accumulate data until the first AOFRW success when AOF is dynamically enabled in tests/integration/aof-multi-part.tcl
       > Expected 'file appendonly.aof.2.base.rdb seq 2 type b' to be equal to 'file appendonly.aof.1.base.rdb seq 1 type b' (context: type source line 125 file /build/redis-7.0.9/tests/support/aofmanifest.tcl cmd {assert_equal [lindex $lines $i] [lindex $content $i]} proc ::assert_aof_manifest_content level 2)
       > Cleanup: may take some time... OK
       For full logs, run 'nix log /nix/store/68a2mzcsidqb8h4kz153c31c0drwj7hm-redis-7.0.9.drv'.
error: builder for '/nix/store/68a2mzcsidqb8h4kz153c31c0drwj7hm-redis-7.0.9.drv' failed with exit code 1
error: 1 dependencies of derivation '/nix/store/bkzshfakd85403dj97dh9c7ark208zm7-python3.10-django-redis-5.2.0.drv' failed to build
error: 1 dependencies of derivation '/nix/store/190g51x83kzjy362yaz7k8z8vhzj513j-netbox-3.4.5.drv' failed to build
error: 1 dependencies of derivation '/nix/store/fap8zc20jsq341yccf2f8ncywdd99gfm-review-shell.drv' failed to build

minijackson · 2023-03-12T16:37:05Z

That's weird, I just rebased, and ran nix build '.#nixosTests.netbox'. I had to rebuild the kernel and redis, but it built without issues...

KiruyaMomochi · 2023-03-22T00:34:17Z

When running the latest nixpkgs with patch from this PR, I can't start netbox successfully.

Mar 22 08:24:33 carmina systemd[1]: Starting NetBox migrations...
Mar 22 08:24:35 carmina netbox[74931]: Operations to perform:
Mar 22 08:24:35 carmina netbox[74931]:   Apply all migrations: admin, auth, circuits, contenttypes, dcim, django_rq, extras, ipam, sessions, social_django, taggit, tenancy, users, virtualization, wireless
Mar 22 08:24:36 carmina netbox[74931]: Running migrations:
Mar 22 08:24:36 carmina netbox[74931]:   No migrations to apply.
Mar 22 08:24:36 carmina netbox[74931]:   Your models in app(s): 'social_django' have changes that are not yet reflected in a migration, and so won't be applied.
Mar 22 08:24:36 carmina netbox[74931]:   Run 'manage.py makemigrations' to make new migrations, and then re-run 'manage.py migrate' to apply them.
Mar 22 08:24:36 carmina systemd[1]: netbox-migration.service: Deactivated successfully.
Mar 22 08:24:36 carmina systemd[1]: Finished NetBox migrations.
Mar 22 08:24:36 carmina systemd[1]: netbox-migration.service: Consumed 2.723s CPU time, no IP traffic.

I guess the error is introduced by the 5.1.0 update of social-auth-app-django. We may need to pin an older version in the package file.

I have tried following NixOS wiki to override social-auth-app-django in an overlay. However, neither overriding netbox or just override python3 package works for me.

Is this because python3 has already be overriden by this package?

https://github.com/NixOS/nixpkgs/blob/297c4f66bafbe65e8b27fe914c9aaa625bce2a18/pkgs/servers/web-apps/netbox/default.nix#L11-L15

stuebinm · 2023-03-26T16:01:16Z

I have tried following NixOS wiki to override social-auth-app-django in an overlay. However, neither overriding netbox or just override python3 package works for me.

Is this because python3 has already be overriden by this package?

https://github.com/NixOS/nixpkgs/blob/297c4f66bafbe65e8b27fe914c9aaa625bce2a18/pkgs/servers/web-apps/netbox/default.nix#L11-L15

It is, this is a limitation of how python's packageOverrides work in nixpkgs. I've run into this issue before, and just opened #223268 with a workaround to allow overrides to work as expected.

Until then, an easy but hacky workaround (without changing nixpkgs) is to also put django = super.django_4 into your own package overrides, and then disable python's overriding functionality entirely so the snippet you linked can't change it back (e.g. by doing let python3 = (python3.override { (your package overrides) }) // { override = _: python3 } in ...).

RaitoBezarius · 2023-03-28T23:50:25Z

3.4.7 is out, can I do something to help you get to the finish line here @minijackson ? I can take over and push the last changes if needed.

nixos/doc/manual/release-notes/rl-2305.section.md

pkgs/servers/web-apps/netbox/3.3.nix

RaitoBezarius · 2023-04-01T20:25:33Z

@minijackson friendly ping so that I can take over if you're too busy right now :)

minijackson · 2023-04-02T08:26:59Z

@RaitoBezarius sure, go ahead. Sorry for the delay...

RaitoBezarius · 2023-04-04T09:44:24Z

@minijackson @SuperSandro2000 please could you give a review to this?

- allows specifying allowed hosts - setup sane default for logging

…ration

reintroduce previous version, use in NixOS module if stateVersion < 23.05

minijackson · 2023-04-05T07:54:07Z

Seems good, thanks! (Can't make this a GitHub approval, since I'm the original author of the PR)

All tests pass on my machine:

nix build -f ./nixpkgs "netbox.tests.netbox"
NIXPKGS_ALLOW_INSECURE=1 nix build -f ./nixpkgs "netbox_3_3.tests.netbox"

Result of nixpkgs-review pr 206983 run on x86_64-linux 1

1 package marked as broken and skipped:

netbox_3_3

2 packages blacklisted:

nixos-install-tools
tests.nixos-functions.nixos-test

1 package built:

netbox

RaitoBezarius · 2023-04-05T12:05:05Z

For the next time, let's try to do smaller PRs and smaller scale changes to get easier merges. :)
Thank you to everyone involved!

github-actions bot added 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: changelog 8.has: documentation This PR adds or changes documentation 8.has: module (update) This PR changes an existing module in `nixos/` labels Dec 20, 2022

ofborg bot requested review from n0emis and RaitoBezarius December 20, 2022 09:47

ofborg bot added 11.by: package-maintainer This PR was created by the maintainer of the package it changes 10.rebuild-darwin: 1-10 10.rebuild-darwin: 1 10.rebuild-linux: 1-10 labels Dec 20, 2022

RaitoBezarius changed the title ~~Netbox 3.3.9 -> 3.4.1~~ netbox: 3.3.9 -> 3.4.1, RFC42-style options, more tests Dec 21, 2022

minijackson force-pushed the netbox-3.4.1 branch from c57d54d to da65aab Compare January 4, 2023 09:19

minijackson changed the title ~~netbox: 3.3.9 -> 3.4.1, RFC42-style options, more tests~~ netbox: 3.3.9 -> 3.4.2, RFC42-style options, more tests Jan 4, 2023

minijackson force-pushed the netbox-3.4.1 branch from da65aab to 2e460c1 Compare January 4, 2023 09:24

minijackson force-pushed the netbox-3.4.1 branch from 2e460c1 to e91c4ae Compare January 21, 2023 15:08

minijackson changed the title ~~netbox: 3.3.9 -> 3.4.2, RFC42-style options, more tests~~ netbox: 3.3.9 -> 3.4.3, RFC42-style options, more tests Jan 21, 2023

RaitoBezarius approved these changes Jan 25, 2023

View reviewed changes

RaitoBezarius reviewed Jan 25, 2023

View reviewed changes

pkgs/servers/web-apps/netbox/default.nix Outdated Show resolved Hide resolved

minijackson force-pushed the netbox-3.4.1 branch from e91c4ae to a17f1ea Compare March 12, 2023 15:16

minijackson requested a review from RaitoBezarius March 12, 2023 15:16

RaitoBezarius changed the title ~~netbox: 3.3.9 -> 3.4.3, RFC42-style options, more tests~~ netbox: 3.3.9 -> 3.4.5, RFC42-style options, more tests Mar 12, 2023

RaitoBezarius approved these changes Mar 12, 2023

View reviewed changes

RaitoBezarius mentioned this pull request Mar 25, 2023

netbox: 3.3.9 -> 3.4.6 #223065

Closed

12 tasks

SuperSandro2000 reviewed Mar 29, 2023

View reviewed changes

nixos/doc/manual/release-notes/rl-2305.section.md Outdated Show resolved Hide resolved

pkgs/servers/web-apps/netbox/3.3.nix Outdated Show resolved Hide resolved

RaitoBezarius force-pushed the netbox-3.4.1 branch from 297c4f6 to 24e26c6 Compare April 4, 2023 09:12

RaitoBezarius force-pushed the netbox-3.4.1 branch from 16d7186 to f4f3a71 Compare April 4, 2023 09:45

minijackson and others added 13 commits April 4, 2023 11:45

netbox: 3.3.9 -> 3.4.1

bc4b08a

formats.pythonVars: init

2cb6dc9

nixos/netbox: RFC42-style options

36a550c

- allows specifying allowed hosts - setup sane default for logging

nixos/tests/netbox: test through proxy, REST API, GraphQL, LDAP integ…

9497639

…ration

nixos/doc: add release notes for NetBox changes

70e95c6

netbox: 3.4.1 -> 3.4.2

52f3031

netbox: 3.4.2 -> 3.4.3

eeb17fc

netbox: 3.4.3 -> 3.4.5

8dbfb9e

netbox: 3.4.5 -> 3.4.6

6e05413

netbox_3_3: init

78eb4d6

reintroduce previous version, use in NixOS module if stateVersion < 23.05

netbox: 3.4.6 -> 3.4.7

c1081bf

netbox: introduce common function for generic packages, mark 3.3.9 EOL

03497d4

netbox_3_3: 3.3.9 -> 3.3.10

e965c5c

RaitoBezarius force-pushed the netbox-3.4.1 branch from f4f3a71 to e965c5c Compare April 4, 2023 09:47

RaitoBezarius changed the title ~~netbox: 3.3.9 -> 3.4.5, RFC42-style options, more tests~~ netbox: 3.3.9 -> 3.4.7, netbox_3_3: init at 3.3.10, RFC42-style options, more tests Apr 5, 2023

RaitoBezarius merged commit a6bc6ed into NixOS:master Apr 5, 2023

minijackson deleted the netbox-3.4.1 branch April 26, 2023 20:00

minijackson mentioned this pull request Aug 21, 2024

Nixpkgs committers interest list #321665

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

netbox: 3.3.9 -> 3.4.7, netbox_3_3: init at 3.3.10, RFC42-style options, more tests #206983

netbox: 3.3.9 -> 3.4.7, netbox_3_3: init at 3.3.10, RFC42-style options, more tests #206983

minijackson commented Dec 20, 2022 •

edited

Loading

RaitoBezarius commented Dec 21, 2022

centromere commented Jan 20, 2023

minijackson commented Jan 21, 2023

RaitoBezarius left a comment

RaitoBezarius Jan 25, 2023

minijackson Mar 12, 2023

RaitoBezarius commented Mar 11, 2023

minijackson commented Mar 12, 2023

RaitoBezarius left a comment

RaitoBezarius commented Mar 12, 2023

RaitoBezarius commented Mar 12, 2023

minijackson commented Mar 12, 2023

KiruyaMomochi commented Mar 22, 2023 •

edited

Loading

stuebinm commented Mar 26, 2023

RaitoBezarius commented Mar 28, 2023

RaitoBezarius commented Apr 1, 2023

minijackson commented Apr 2, 2023

RaitoBezarius commented Apr 4, 2023

minijackson commented Apr 5, 2023

RaitoBezarius commented Apr 5, 2023

netbox: 3.3.9 -> 3.4.7, netbox_3_3: init at 3.3.10, RFC42-style options, more tests #206983

netbox: 3.3.9 -> 3.4.7, netbox_3_3: init at 3.3.10, RFC42-style options, more tests #206983

Conversation

minijackson commented Dec 20, 2022 • edited Loading

Description of changes

Things done

RaitoBezarius commented Dec 21, 2022

centromere commented Jan 20, 2023

minijackson commented Jan 21, 2023

RaitoBezarius left a comment

Choose a reason for hiding this comment

RaitoBezarius Jan 25, 2023

Choose a reason for hiding this comment

minijackson Mar 12, 2023

Choose a reason for hiding this comment

RaitoBezarius commented Mar 11, 2023

minijackson commented Mar 12, 2023

RaitoBezarius left a comment

Choose a reason for hiding this comment

RaitoBezarius commented Mar 12, 2023

RaitoBezarius commented Mar 12, 2023

minijackson commented Mar 12, 2023

KiruyaMomochi commented Mar 22, 2023 • edited Loading

stuebinm commented Mar 26, 2023

RaitoBezarius commented Mar 28, 2023

RaitoBezarius commented Apr 1, 2023

minijackson commented Apr 2, 2023

RaitoBezarius commented Apr 4, 2023

minijackson commented Apr 5, 2023

RaitoBezarius commented Apr 5, 2023

minijackson commented Dec 20, 2022 •

edited

Loading

KiruyaMomochi commented Mar 22, 2023 •

edited

Loading