unify Config generation, loading and updating #1586

nazarewk · 2024-02-15T11:09:36Z

Describe your changes

There are 2 primary changes:

(misc) allows passing custom system service name (helps nixos/netbird: harden and extend options NixOS/nixpkgs#287236 )
unifies configuration loading and updating into a single Config.apply(ConfigInput):
- has better logging (informs of everything it does)
- accepts both empty Config{} and empty InputConfig{} filling in missing parts automatically
- handling was slightly inconsistent between createNewConfig() and update()
- added InterfaceBlacklist to the InputConfig

Basically when I tried to start the netbird service run with just WgIface and WgPort present in config.json it crashed, so I got to fixing stuff :)

Issue ticket number and link

partially addresses NB_{WIREGUARD_PORT,INTERFACE_NAME} is not supported by anything except netbird up #1562 by making incomplete configs acceptable (currently it segfaults on missing pointers/values)
to be used in nixos/netbird: harden and extend options NixOS/nixpkgs#287236

Checklist

Is it a bug fix
Is a typo/documentation fix
Is a feature enhancement
It is a refactor
Created tests that fail without the change (if possible)
Extended the README / documentation, if necessary

nazarewk · 2024-02-15T11:11:45Z

Note I was unable to run tests locally so didn't touch those hoping nothing will break

kdn@krul ~/d/g/n/netbird (fixups) [1]> go test -exec sudo ./...
package github.com/netbirdio/netbird/client/ui
        imports fyne.io/fyne/v2/app
        imports fyne.io/fyne/v2/internal/driver/glfw
        imports fyne.io/fyne/v2/internal/driver/common
        imports fyne.io/fyne/v2/internal/painter/gl
        imports github.com/go-gl/gl/v3.2-core/gl: build constraints exclude all Go files in /home/kdn/go/pkg/mod/github.com/go-gl/gl@v0.0.0-20210813123233-e4099ee2221f/v3.2-core/gl

nazarewk · 2024-02-15T11:42:02Z

this commit seems to work on a live system dcf9b84

before

don't mind that jq error, it was still creating a file and starting:

Feb 15 09:44:54 krul (re-start)[420986]: netbird.service: ConfigurationDirectory 'netbird' already exists but the mode is different. (File system: 755 ConfigurationDirectoryMode: 700)
Feb 15 09:44:54 krul netbird-pre-start[420986]: created empty /var/lib/netbird/config.json
Feb 15 09:44:54 krul netbird-pre-start[420999]: jq: error: syntax error, unexpected '/', expecting end of file (Unix shell quoting issues?) at <top-level>, line 1:
Feb 15 09:44:54 krul netbird-pre-start[420999]: /var/lib/netbird/config.json
Feb 15 09:44:54 krul netbird-pre-start[420999]: jq: 1 compile error
Feb 15 09:44:54 krul netbird-pre-start[421001]: 0a1,4
Feb 15 09:44:54 krul netbird-pre-start[421001]: > {
Feb 15 09:44:54 krul netbird-pre-start[421001]: >   "WgIface": "wt0",
Feb 15 09:44:54 krul netbird-pre-start[421001]: >   "WgPort": 51831
Feb 15 09:44:54 krul netbird-pre-start[421001]: > }
Feb 15 09:44:54 krul systemd[1]: Started A WireGuard-based mesh network that connects your devices into a single private network.
Feb 15 09:44:54 krul (bird-wt0)[421033]: netbird.service: ConfigurationDirectory 'netbird' already exists but the mode is different. (File system: 755 ConfigurationDirectoryMode: 700)
Feb 15 09:44:54 krul netbird-wt0[421033]: 2024-02-15T09:44:54+01:00 INFO client/cmd/service_controller.go:24: starting Netbird service
Feb 15 09:44:54 krul netbird-wt0[421033]: panic: runtime error: invalid memory address or nil pointer dereference
Feb 15 09:44:54 krul netbird-wt0[421033]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0xf0997d]
Feb 15 09:44:54 krul netbird-wt0[421033]: goroutine 8 [running]:
Feb 15 09:44:54 krul netbird-wt0[421033]: net/url.(*URL).Hostname(...)
Feb 15 09:44:54 krul netbird-wt0[421033]:         net/url/url.go:1144
Feb 15 09:44:54 krul netbird-wt0[421033]: github.com/netbirdio/netbird/client/internal.UpdateOldManagementURL({0x16b78e8, 0xc0000e65f0}, 0xc0008266e0, {0xc00004a03a, 0x1c})
Feb 15 09:44:54 krul netbird-wt0[421033]:         github.com/netbirdio/netbird/client/internal/config.go:358 +0xfd
Feb 15 09:44:54 krul netbird-wt0[421033]: github.com/netbirdio/netbird/client/server.(*Server).Start(0xc000002480)
Feb 15 09:44:54 krul netbird-wt0[421033]:         github.com/netbirdio/netbird/client/server/server.go:109 +0x4e5
Feb 15 09:44:54 krul netbird-wt0[421033]: github.com/netbirdio/netbird/client/cmd.(*program).Start.func1()
Feb 15 09:44:54 krul netbird-wt0[421033]:         github.com/netbirdio/netbird/client/cmd/service_controller.go:59 +0x513
Feb 15 09:44:54 krul netbird-wt0[421033]: created by github.com/netbirdio/netbird/client/cmd.(*program).Start in goroutine 1
Feb 15 09:44:54 krul netbird-wt0[421033]:         github.com/netbirdio/netbird/client/cmd/service_controller.go:47 +0x315
Feb 15 09:44:54 krul systemd[1]: netbird.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Feb 15 09:44:54 krul systemd[1]: netbird.service: Failed with result 'exit-code'.
Feb 15 09:44:54 krul systemd[1]: netbird.service: Scheduled restart job, restart counter is at 1.

after

kdn@krul ~/d/g/n/nix-configs (main)> journalctl --since -5m -u netbird.service
Feb 15 12:35:10 krul systemd[1]: Stopping A WireGuard-based mesh network that connects your devices into a single private network...
Feb 15 12:35:10 krul netbird-wt0[647207]: 2024-02-15T12:35:10+01:00 INFO client/cmd/root.go:150: shutdown signal received
Feb 15 12:35:12 krul netbird-wt0[647207]: 2024-02-15T12:35:12+01:00 INFO client/cmd/service_controller.go:80: stopped Netbird service
Feb 15 12:35:12 krul systemd[1]: netbird.service: Deactivated successfully.
Feb 15 12:35:12 krul systemd[1]: Stopped A WireGuard-based mesh network that connects your devices into a single private network.
Feb 15 12:35:12 krul systemd[1]: Starting A WireGuard-based mesh network that connects your devices into a single private network...
Feb 15 12:35:12 krul netbird-pre-start[721471]: Comparing /var/lib/netbird/config.json with /var/lib/netbird/config.json.new ...
Feb 15 12:35:12 krul netbird-pre-start[721474]: 52c52
Feb 15 12:35:12 krul netbird-pre-start[721474]: <   "WgPort": 51820
Feb 15 12:35:12 krul netbird-pre-start[721474]: ---
Feb 15 12:35:12 krul netbird-pre-start[721474]: >   "WgPort": 51831
Feb 15 12:35:12 krul netbird-pre-start[721471]: Updating /var/lib/netbird/config.json ...
Feb 15 12:35:12 krul systemd[1]: Started A WireGuard-based mesh network that connects your devices into a single private network.
Feb 15 12:35:12 krul netbird-wt0[721477]: 2024-02-15T12:35:12+01:00 INFO client/cmd/service_controller.go:24: starting Netbird service
Feb 15 12:35:12 krul netbird-wt0[721477]: 2024-02-15T12:35:12+01:00 INFO client/cmd/service_controller.go:64: started daemon server: /var/run/netbird/sock
Feb 15 12:35:12 krul netbird-wt0[721477]: 2024-02-15T12:35:12+01:00 INFO client/internal/connect.go:95: starting NetBird client version 0.25.7
Feb 15 12:35:12 krul netbird-wt0[721477]: 2024-02-15T12:35:12+01:00 ERRO client/server/server.go:121: init connections: rpc error: code = PermissionDenied desc = no peer auth method provided, please use a setup key or interactive SSO login

nazarewk · 2024-03-11T09:33:02Z

moved service name flag out into #1691

nazarewk · 2024-04-09T08:55:37Z

FYI: this seems like a good standard for configurations https://uapi-group.org/specifications/specs/configuration_files_specification/

…pdate() as a bonus it ensures returned Config object doesn't have any configuration values missing

# Conflicts: # client/internal/config.go

mlsmaycon

Thanks @nazarewk for the PR. I've resolved some merge conflict and fixed a small issue with nat external IPs

nazarewk force-pushed the fixups branch from 2dfabe7 to a49d1d7 Compare February 15, 2024 11:24

nazarewk force-pushed the fixups branch from a49d1d7 to dcf9b84 Compare February 15, 2024 11:30

nazarewk changed the title ~~add --service/-s flag for specifying system service name~~ unify Config generation and loading Feb 15, 2024

nazarewk changed the title ~~unify Config generation and loading~~ unify Config generation, loading and updating Feb 15, 2024

nazarewk mentioned this pull request Feb 15, 2024

nixos/netbird: harden and extend options NixOS/nixpkgs#287236

Open

13 tasks

nazarewk force-pushed the fixups branch 2 times, most recently from 7b1c7ff to 1b7da57 Compare February 26, 2024 09:29

nazarewk mentioned this pull request Mar 11, 2024

add --service/-s flag for specifying system service name #1691

Merged

6 tasks

nazarewk force-pushed the fixups branch from 1b7da57 to 1a7e205 Compare March 11, 2024 09:32

hoozecn mentioned this pull request Mar 22, 2024

support to configure extra blacklist of iface in "up" command #1734

Merged

config.go: pull unified Config.apply() out of createNewConfig() and u…

62f6c77

…pdate() as a bonus it ensures returned Config object doesn't have any configuration values missing

nazarewk force-pushed the fixups branch from 1a7e205 to 62f6c77 Compare April 10, 2024 06:47

mlsmaycon added 3 commits May 8, 2024 17:28

Merge branch 'main' into fixups

41d0374

# Conflicts: # client/internal/config.go

add network monitor config update

f215d1b

fix nat external IPs check

48a445a

mlsmaycon approved these changes May 8, 2024

View reviewed changes

mlsmaycon merged commit ceee421 into netbirdio:main May 8, 2024
16 checks passed

nazarewk deleted the fixups branch May 8, 2024 17:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

unify Config generation, loading and updating #1586

unify Config generation, loading and updating #1586

nazarewk commented Feb 15, 2024 •

edited

Loading

nazarewk commented Feb 15, 2024 •

edited

Loading

nazarewk commented Feb 15, 2024

nazarewk commented Mar 11, 2024

nazarewk commented Apr 9, 2024

mlsmaycon left a comment

unify Config generation, loading and updating #1586

unify Config generation, loading and updating #1586

Conversation

nazarewk commented Feb 15, 2024 • edited Loading

Describe your changes

Issue ticket number and link

Checklist

nazarewk commented Feb 15, 2024 • edited Loading

nazarewk commented Feb 15, 2024

nazarewk commented Mar 11, 2024

nazarewk commented Apr 9, 2024

mlsmaycon left a comment

Choose a reason for hiding this comment

nazarewk commented Feb 15, 2024 •

edited

Loading

nazarewk commented Feb 15, 2024 •

edited

Loading