Multiple BirdNET-Pi's on a single local network

[lukemodular]

Thanks for this great project!

I'm trying to setup a network of 3 BirdNET-Pi's scattered in a large park area.
I would like to interface them individually on the same local network.
The idea is to configure them with static IP's and connect them to an ethernet-switch.
So far I had no luck configuring individual URL's in the birdnet.conf or in the caddyfile.
I also tried to access them directly via their IP's.
This way I managed to access their different services on the respective ports (eg. the extraction log on 10.10.10.10:8080, etc), but unfortunately not the basic webinterface (10.10.10.10:80 doesn't seem to work).

Any ideas how to set this up properly?

[mcguirepr89]

Hello, @lukemodular!! Thank you so much for your interest in BirdNET-Pi -- I noticed you starred the project a few days back, so thank you also for that!

I'm trying to setup a network of 3 BirdNET-Pi's scattered in a large park area.

Super cool! And what a lucky park! I would love if you were able to share some of your findings :)

The idea is to configure them with static IP's and connect them to an ethernet-switch.
So far I had no luck configuring individual URL's in the birdnet.conf or in the caddyfile.
I also tried to access them directly via their IP's.
This way I managed to access their different services on the respective ports (eg. the extraction log on 10.10.10.10:8080, etc), but unfortunately not the basic webinterface (10.10.10.10:80 doesn't seem to work).

This approach is a good approach. You can follow the steps below to (untested but should work) get this working for you. The steps below assume that each BirdNET-Pi has its own static IP address configured (OR if you wanted to setup DHCP reservations on the network switch, that should also suffice.)

So far I had no luck configuring individual URL's in the birdnet.conf or in the caddyfile.

The Caddyfile is the trick here. It only listens for "http://birdnetpi.local" be default, but you can configure it to use its IP address instead.
To do this, (again, this is untested, but should work), edit the birdnet.conf file's URL entries as it sounds like you have attempted. Instead of a URL, though, simply put the static IP address for all three custom URLs. After setting those variables, you will need to run an update (from the command line since you cannot reach the web interface) via update_birdnet.sh.

update_birdnet.sh will re-install the system with whatever it finds in birdnet.conf. So, the idea is that the update will rewrite the Caddyfile so that it will also listen on the BirdNET-Pi's static IP address.

I am sorry to provide untested instructions, but I know we will get this working for you whether the steps above achieve it immediately or whether a little more work may be needed.

I'm around today and happy to help more, so just let me know how it's going and we can take it from there

MY best!

[lukemodular]

Dear Patrick, thanks for the super fast reply and your offer to help!

I followed your instructions on a fresh install of BirdNET-Pi.
Unfortunately I get still the same results:
No website directly on the IP, but :8080, :8000 etc. are working correctly.
Weirdly enough, also birdnetpi.local doesn't seem to be working anymore.

Here you can find the entries in the /etc/caddy/Caddyfile (I also tried the ip without the http://) :

http://localhost http://birdnetpi.local http://192.168.137.12 {
root * /home/pi/BirdSongs/Extracted
file_server browse
basicauth /Processed* {
birdnet XXXX
}
basicauth /scripts* {
birdnet XXXX
}
basicauth /stream {
birdnet XXXX
}
basicauth /phpsysinfo* {
birdnet XXXX
}
reverse_proxy /stream localhost:8000
php_fastcgi unix//run/php/php7.4-fpm.sock
}

http://192.168.137.12 {
reverse_proxy localhost:8888
}

http://192.168.137.12 {
reverse_proxy localhost:8080
}

And here the configuration of the static ip in /etc/dhcpcd.conf
interface eth0
static ip_address=192.168.137.12/24
static routers=192.168.137.1
static domain_name_servers=192.168.137.1

PS:
I would love to share you our findings of this project!
It is actually for a VR art installation in the park of an art center in central France by sculptor Marie Lelouche, which is reacting to the presence of different birds.

[mcguirepr89]

@lukemodular First of all:

It is actually for a VR art installation in the park of an art center in central France by sculptor Marie Lelouche, which is reacting to the presence of different birds.

This is SOOOOOOOOOOOOO awesome. My partner is an independent art writer and an editor-at-large for a US Southern arts publication, Burnaway, so I am thrilled to hear that this is going to be used for art!!!!

Could you send me the last 200 hundred (there may not be that many) lines of the journal log for caddy?

journalctl --no-pager -n200 -u caddy

To keep that log formatting in your Github comment, add ``` on the first line before pasting the output on the next line, and then add another ``` as the last line to close out the log formatting.
For example,
```
Hello
```
would generate:

Hello

After getting those logs, it may be worth trying the process again using only the static IP addresses with port 80 appended, so your birdnet.conf would look like:

BIRDNETPI_URL=192.168.137.12:80
EXTRACTIONLOG_URL=192.168.137.12:80
BIRDNETLOG_URL=192.168.137.12:80

Although I don't really think that will make much of a difference, it is something that I would try myself were I to be directly troubleshooting from the command line.

After I've had a chance to look through the caddy logs, I will let you know what the best way forward will be. (There are many ways to make this work, I'm just trying to be sure we proceed gradually so as not to have to back track troubleshooting too much.)

Fingers crossed! (running some errands, but will be back later to help more)

[lukemodular]

Thanks for the link to your partner's work, that's super interesting!

Ah, there seem to be a problem, caddy doesn't works properly (also after changing to 192.168.137.12:80) :

Dec 03 18:29:35 birdnetpi systemd[1]: Starting Caddy...
Dec 03 18:29:35 birdnetpi caddy[3951]: caddy.HomeDir=/var/lib/caddy
Dec 03 18:29:35 birdnetpi caddy[3951]: caddy.AppDataDir=/var/lib/caddy/.local/share/caddy
Dec 03 18:29:35 birdnetpi caddy[3951]: caddy.AppConfigDir=/var/lib/caddy/.config/caddy
Dec 03 18:29:35 birdnetpi caddy[3951]: caddy.ConfigAutosavePath=/var/lib/caddy/.config/caddy/autosave.json
Dec 03 18:29:35 birdnetpi caddy[3951]: caddy.Version=v2.4.6 h1:HGkGICFGvyrodcqOOclHKfvJC0qTU7vny/7FhYp9hNw=
Dec 03 18:29:35 birdnetpi caddy[3951]: runtime.GOOS=linux
Dec 03 18:29:35 birdnetpi caddy[3951]: runtime.GOARCH=arm64
Dec 03 18:29:35 birdnetpi caddy[3951]: runtime.Compiler=gc
Dec 03 18:29:35 birdnetpi caddy[3951]: runtime.NumCPU=4
Dec 03 18:29:35 birdnetpi caddy[3951]: runtime.GOMAXPROCS=4
Dec 03 18:29:35 birdnetpi caddy[3951]: runtime.Version=go1.17.2
Dec 03 18:29:35 birdnetpi caddy[3951]: os.Getwd=/
Dec 03 18:29:35 birdnetpi caddy[3951]: LANG=en_GB.UTF-8
Dec 03 18:29:35 birdnetpi caddy[3951]: LC_CTYPE=en_US.UTF-8
Dec 03 18:29:35 birdnetpi caddy[3951]: LC_MESSAGES=en_US.UTF-8
Dec 03 18:29:35 birdnetpi caddy[3951]: PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Dec 03 18:29:35 birdnetpi caddy[3951]: NOTIFY_SOCKET=/run/systemd/notify
Dec 03 18:29:35 birdnetpi caddy[3951]: HOME=/var/lib/caddy
Dec 03 18:29:35 birdnetpi caddy[3951]: LOGNAME=caddy
Dec 03 18:29:35 birdnetpi caddy[3951]: USER=caddy
Dec 03 18:29:35 birdnetpi caddy[3951]: INVOCATION_ID=7be80a7af4dc422095894fc7ee9cc41b
Dec 03 18:29:35 birdnetpi caddy[3951]: JOURNAL_STREAM=8:25002
Dec 03 18:29:35 birdnetpi caddy[3951]: {"level":"info","ts":1638552575.7955518,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":""}
Dec 03 18:29:35 birdnetpi caddy[3951]: run: adapting config using caddyfile: duplicate site address not allowed: 'http://192.168.137.12' in [http://192.168.137.12] (site block 1, key 0)
Dec 03 18:29:35 birdnetpi systemd[1]: caddy.service: Main process exited, code=exited, status=1/FAILURE
Dec 03 18:29:35 birdnetpi systemd[1]: caddy.service: Failed with result 'exit-code'.
Dec 03 18:29:35 birdnetpi systemd[1]: Failed to start Caddy.
Dec 03 18:29:38 birdnetpi systemd[1]: caddy.service: Unit cannot be reloaded because it is inactive.

http://localhost http://birdnetpi.local http://192.168.137.12 {
  root * /home/pi/BirdSongs/Extracted
  file_server browse
  basicauth /Processed* {
    birdnet JDJhJDE0JFBILkl3Rm9ESmZQL3NFWDRtZlhxYS5idDFYL2RJdXAvNWFHa3ZadW1rYktQRS5LR0E4Wml1
  }
  basicauth /scripts* {
    birdnet JDJhJDE0JFBILkl3Rm9ESmZQL3NFWDRtZlhxYS5idDFYL2RJdXAvNWFHa3ZadW1rYktQRS5LR0E4Wml1
  }
  basicauth /stream {
    birdnet JDJhJDE0JFBILkl3Rm9ESmZQL3NFWDRtZlhxYS5idDFYL2RJdXAvNWFHa3ZadW1rYktQRS5LR0E4Wml1
  }
  basicauth /phpsysinfo* {
    birdnet JDJhJDE0JFBILkl3Rm9ESmZQL3NFWDRtZlhxYS5idDFYL2RJdXAvNWFHa3ZadW1rYktQRS5LR0E4Wml1
  }
  reverse_proxy /stream localhost:8000
  php_fastcgi unix//run/php/php7.4-fpm.sock
}
http://192.168.137.12 {
  reverse_proxy localhost:8888
}
http://192.168.137.12 {
  reverse_proxy localhost:8080
}

Thank you so much for your help!

[lukemodular]

Some further investigations:
Cleaning the Caddyfile manually and sudo systemctl restart caddy helped!
The main page is now working on http://192.168.137.12
Some links are now unfortunately broken, for example the one for scripts is shown as http://192.168.137.12/192.168.137.12:80/scripts/
But it works manually on: http://192.168.137.12/scripts/

http://localhost http://birdnetpi.local http://192.168.137.12 {
  root * /home/pi/BirdSongs/Extracted
  file_server browse
  basicauth /Processed* {
    birdnet JDJhJDE0JFBILkl3Rm9ESmZQL3NFWDRtZlhxYS5idDFYL2RJdXAvNWFHa3ZadW1rYktQRS5LR0E4Wml1
  }
  basicauth /scripts* {
    birdnet JDJhJDE0JFBILkl3Rm9ESmZQL3NFWDRtZlhxYS5idDFYL2RJdXAvNWFHa3ZadW1rYktQRS5LR0E4Wml1
  }
  basicauth /stream {
    birdnet JDJhJDE0JFBILkl3Rm9ESmZQL3NFWDRtZlhxYS5idDFYL2RJdXAvNWFHa3ZadW1rYktQRS5LR0E4Wml1
  }
  basicauth /phpsysinfo* {
    birdnet JDJhJDE0JFBILkl3Rm9ESmZQL3NFWDRtZlhxYS5idDFYL2RJdXAvNWFHa3ZadW1rYktQRS5LR0E4Wml1
  }
  reverse_proxy /stream localhost:8000
  php_fastcgi unix//run/php/php7.4-fpm.sock
}

Dec 03 18:59:22 birdnetpi systemd[1]: Starting Caddy...
Dec 03 18:59:22 birdnetpi caddy[6343]: caddy.HomeDir=/var/lib/caddy
Dec 03 18:59:22 birdnetpi caddy[6343]: caddy.AppDataDir=/var/lib/caddy/.local/share/caddy
Dec 03 18:59:22 birdnetpi caddy[6343]: caddy.AppConfigDir=/var/lib/caddy/.config/caddy
Dec 03 18:59:22 birdnetpi caddy[6343]: caddy.ConfigAutosavePath=/var/lib/caddy/.config/caddy/autosave.json
Dec 03 18:59:22 birdnetpi caddy[6343]: caddy.Version=v2.4.6 h1:HGkGICFGvyrodcqOOclHKfvJC0qTU7vny/7FhYp9hNw=
Dec 03 18:59:22 birdnetpi caddy[6343]: runtime.GOOS=linux
Dec 03 18:59:22 birdnetpi caddy[6343]: runtime.GOARCH=arm64
Dec 03 18:59:22 birdnetpi caddy[6343]: runtime.Compiler=gc
Dec 03 18:59:22 birdnetpi caddy[6343]: runtime.NumCPU=4
Dec 03 18:59:22 birdnetpi caddy[6343]: runtime.GOMAXPROCS=4
Dec 03 18:59:22 birdnetpi caddy[6343]: runtime.Version=go1.17.2
Dec 03 18:59:22 birdnetpi caddy[6343]: os.Getwd=/
Dec 03 18:59:22 birdnetpi caddy[6343]: LANG=en_GB.UTF-8
Dec 03 18:59:22 birdnetpi caddy[6343]: LC_CTYPE=en_US.UTF-8
Dec 03 18:59:22 birdnetpi caddy[6343]: LC_MESSAGES=en_US.UTF-8
Dec 03 18:59:22 birdnetpi caddy[6343]: PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Dec 03 18:59:22 birdnetpi caddy[6343]: NOTIFY_SOCKET=/run/systemd/notify
Dec 03 18:59:22 birdnetpi caddy[6343]: HOME=/var/lib/caddy
Dec 03 18:59:22 birdnetpi caddy[6343]: LOGNAME=caddy
Dec 03 18:59:22 birdnetpi caddy[6343]: USER=caddy
Dec 03 18:59:22 birdnetpi caddy[6343]: INVOCATION_ID=a78556acfbba4f288fc5afda60e8fbdb
Dec 03 18:59:22 birdnetpi caddy[6343]: JOURNAL_STREAM=8:28323
Dec 03 18:59:22 birdnetpi caddy[6343]: {"level":"info","ts":1638554362.759031,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":""}
Dec 03 18:59:22 birdnetpi caddy[6343]: {"level":"warn","ts":1638554362.7666748,"msg":"input is not formatted with 'caddy fmt'","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":2}
Dec 03 18:59:22 birdnetpi caddy[6343]: {"level":"info","ts":1638554362.7718089,"logger":"admin","msg":"admin endpoint started","address":"tcp/localhost:2019","enforce_origin":false,"origins":["localhost:2019","[::1]:2019","127.0.0.1:2019"]}
Dec 03 18:59:22 birdnetpi caddy[6343]: {"level":"info","ts":1638554362.7724721,"logger":"http","msg":"server is listening only on the HTTP port, so no automatic HTTPS will be applied to this server","server_name":"srv0","http_port":80}
Dec 03 18:59:22 birdnetpi caddy[6343]: {"level":"info","ts":1638554362.7725105,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x40003aa690"}
Dec 03 18:59:32 birdnetpi caddy[6343]: {"level":"info","ts":1638554372.719127,"logger":"tls","msg":"cleaning storage unit","description":"FileStorage:/var/lib/caddy/.local/share/caddy"}
Dec 03 18:59:32 birdnetpi caddy[6343]: {"level":"info","ts":1638554372.7194035,"logger":"tls","msg":"finished cleaning storage units"}
Dec 03 18:59:32 birdnetpi caddy[6343]: {"level":"info","ts":1638554372.7198818,"msg":"autosaved config (load with --resume flag)","file":"/var/lib/caddy/.config/caddy/autosave.json"}
Dec 03 18:59:32 birdnetpi systemd[1]: Started Caddy.

[mcguirepr89]

Hey, that's great!!!

Your findings make total sense, and my suggestions were a little askew. But I'm glad you've been able to reach the main web-page.

I should have foreseen the issue with using the same IP address for all three URLs in the birdnet.conf file since it would make multiple site-entries with the same address in the Caddyfile, which quickly breaks things.

I'm home now and have an idea for a very clean approach to this for you that will repair those broken links and get you back to using hostnames for URLs. I will test first to see that my idea works and will get back to you soon.

[mcguirepr89]

@lukemodular I have the answer now -- the best answer that is quite elegant, if I may say so myself! And it's tested :)

You (very likely) will be able to achieve the desired results without any need for static IP addresses nor DHCP reservations.

The idea is that you will tell each BirdNET-Pi to announce itself (without regard to its actual hostname) with custom avahi-aliases, and that will be where caddy will bind!

This is also very easy to do in that it will require no manual customization to the Caddyfile.

Do the following on each BirdNET-Pi, changing the obvious hostname numbering where needed (you'll see what I mean below):

1. Issue

   sudo systemctl disable --now avahi-alias@birdnetpi.local.service
   

   This will tell the BirdNET-Pi to stop announcing itself as "birdnetpi.local" on the network.
2. Issue

   sudo systemctl enable --now avahi-alias@birdnetpi1.local.service
   

   This will tell the first BirdNET-Pi to announce itself on the network (LAN and loopback) as "birdnetpi1.local". This means we need to tell caddy to bind to requests that come in for "birdnetpi1.local"
3. To reconfigure Caddy, as was mentioned in previous instructions, simply edit the birdnet.conf file's custom URLs, this time using ONLY "http://birdnetpi1.local" and leaving the other URL entries blank, as below:

   BIRDNETPI_URL=http://birdnetpi1.local
   EXTRACTIONLOG_URL=
   BIRDNETLOG_URL=
   

4. After editing birdnet.conf, run update_birdnet.sh to let the system reconfigure for the new URL

Just repeat those steps on each BirdNET-Pi, incrementing the URL digit in step 2 as many times as there are BirdNET-Pis on the network :)

[mcguirepr89]

I forgot to mention that your updated version of caddy (2.4.6) has a bug with symbolic links, on which BirdNET-Pi relies heavily. To fix this, on each BirdNET-Pi, issue sudo apt install --reinstall caddy=2.4.5 && sudo apt-mark hold caddy.
That will downgrade caddy to the latest known-working version and will put a hold on upgrading it so that you can update the system normally without worrying about updates breaking caddy. And if you are using the pre-installed BirdNET-Pi_v0.9 image, be sure to issue

sudo sed -i 's/ExecStart=.*/ExecStart=echo/g' /etc/systemd/system/network-online.target.wants/systemd-networkd-wait-online.service && sudo reboot

to fix the network bug that causes problems when booting.

[lukemodular]

Many thanks!
Your elegant solution worked real quick, but after updating caddy, it mysteriously broke somehow.
I didn't get it to work again.

I'm back using the fixed ip's.
At least this works perfectly now with the downgraded caddy (links and all).

In any case I will need to stick with the fixed ip's for the other stuff going on in the installation (OSC between the RPI's and Unity), so I will keep with the brute force solution for now ;).

I will keep you posted after testing everything!

[mcguirepr89]

Your elegant solution worked real quick, but after updating caddy, it mysteriously broke somehow.
I didn't get it to work again.

Bummer! If you want to go back to the "elegant solution," you can very likely achieve it again by going through the process once more. I suspect caddy's downgrade replaced the Caddyfile with a default Caddyfile listening only to localhost. If that theory is correct, then another update_birdnet.sh after that caddy downgrade may get you back to using hostnames (avahi-aliases).

I'm back using the fixed ip's.
At least this works perfectly now with the downgraded caddy (links and all).

But with that said, if it works, it works!

Please let me know if you have any questions about how things are put together or if you have any ideas you'd like to just run by me in case I might be able to tell you more about how BirdNET-Pis operate and can be reconfigured. I am also more than happy to help with any additional features you might need these Pis to do. Please do consider these offers as I have 10 days before I start my new job and contributing to the arts and BirdNET-Pi at the same time is a dream come true.

My best regards to you and Marie Lalouche

[lukemodular]

Thank you so much, Patrick!
We'll definitely come back to you if there should be any problems.
For now, everything is up and running with three nodes :)!

Many thanks from Marie as well!
And say hello from both of us to Susan too!

[oliverk71]

This could be of interest for my birdnet-pis (at larger areas) in future! To have several birdnet-pis which recognitions are combined but also available for each pi - would be awesome.
@lukemodular May I have a look at your installation's homepage? I guess you have a top level domain for it, now. Would you like to share it?

[mcguirepr89]

If I may, @lukemodular's wonderful use of a BirdNET-Pi network is at use in the exhibition Out Of Spaces by French artist Marie Lelouche -- their application of BirdNET is truly quite different from the average BirdNET-Pi user. I hope that will suffice as a look at Lukas's/Marie's homepage!

Also, the new additions to the forms branch support multi-node networks through hostnames. Now, when burning the SD card with the RaspiOS-Lite-ARM64 image, you can specify the hostname you'd like the Raspberry Pi to have. Then, your installation will take on that hostname as its configured reachable URL -- so everything that would normally be "birdnetpi.local" will become "yourhostname.local" instead. If left as the default "raspberrypi" then the installation will also use its default "birdnetpi.local".

I hope that is helpful! Let me know if you have any questions.
My best regards,
Patrick

[lukemodular]

Thanks for sharing the link to the exhibition, Patrick!

The pi-network itself is only working locally for now.
But we will change that in the future to share the classified birds!

Great solution to use the rapsi's hostname for the mutli-node network!
That should make the setup super easy.