Solar Version with LTE

[gitalm]

Dear Patrick,

my project is slowly evolving.

• Solar powered with OffGridTec and USB
• I created a GitHub to upload the Data

-BirdDB.csv, IdentifiedSoFar.txt, and folder By_Commmon name is transfered

• I analyze with Observable
• I have a Vodafone LTE Stick K5610 (usb-modeswitch is running)
• I have a WIttyPi to start and end the RPi at 6:00 to 10:00 and have a clock

This solution won't be a LiveBox, but you can see every half an hour the actual data. So i decided for less data to be transfered, but this will be availabe from the Internet (GitHub Sever and Observabel Website) and not from a server running on the wonderful BirdNetPI. I just use the Analyze of the live data and the Audio segments. Thank to 0.11.1 it is now a small *.mp3.

I still have some problems:

• How can i stop all unnecessary services for this setup? e.g. Icecast, ...
• Where do you start the Cronjob?

My Cronjob isn't executed, but there are succesfull Cronjobs (see output). If i manually start the script it is working like desired.

Thanks for your incredible job and help. I atteched all output from the console.

Johannes

Crontab

crontab -l

m h dom mon dow command

30 6-10 * * * sh /home/pi/birdnetpi_github/birdnetpi_github.sh

grep CRON /var/log/syslog
...
Feb 20 16:51:01 birdnetpi CRON[182683]: (pi) CMD (/usr/local/bin/cleanup.sh &> /dev/null)
Feb 20 16:54:01 birdnetpi CRON[184361]: (pi) CMD (/usr/local/bin/cleanup.sh &> /dev/null)
Feb 20 16:55:01 birdnetpi CRON[184882]: (pi) CMD (/usr/local/bin/disk_check.sh &> /dev/null)
Feb 20 16:57:01 birdnetpi CRON[185929]: (pi) CMD (/usr/local/bin/cleanup.sh &> /dev/null)

GitHub

#!/bin/bash

#BirdDB contains alls identified Birds
cp /home/pi/BirdNET-Pi/BirdDB.txt /home/pi/birdnetpi_github/BirdDB.csv
sed -i 's/;/,/g' /home/pi/birdnetpi_github/BirdDB.csv
cp /home/pi/BirdNET-Pi/IdentifiedSoFar.txt /home/pi/birdnetpi_github/
echo "Textfiles copied"

#Copy Audio-Files
cp -aru /home/pi/BirdSongs/Extracted/By_Common_Name/* /home/pi/birdnetpi_github/By_Common_Name
echo "all transfered"

#Commit to the internet
cd /home/pi/birdnetpi_github/
git add --all
git commit -am "BirdNET-PI Solar"
git push -u origin master

WittyPi

================================================================================
| |
| Witty Pi - Realtime Clock + Power Management for Raspberry Pi |
| |
| < Version 2.67 > by UUGear s.r.o. |
| |
================================================================================

Current temperature: 22.50°C / 72.5°F
Your system time is: Fri 25 Feb 2022 08:20:33 CET
Your RTC time is: Fri 25 Feb 2022 08:20:34 CET
Now you can:

1. Write system time to RTC
2. Write RTC time to system
3. Synchronize time
4. Schedule next shutdown [25 11:00:00]
5. Schedule next startup [26 06:00:00]
6. Choose schedule script [in use]
7. Reset data...
8. Exit
   What do you want to do? (1~8)

Vodafone

Should work, but therefore the Cronjob is not running, i wont have access to check.

wwan0: flags=4291<UP,BROADCAST,RUNNING,NOARP,MULTICAST> mtu 1500
inet 169.254.54.210 netmask 255.255.0.0 broadcast 169.254.255.255
inet6 fe80::38d2:9b08:dd5f:42f0 prefixlen 64 scopeid 0x20
ether fe:03:d7:0b:85:8b txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 132 bytes 39167 (38.2 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

WittyPi starts at 6 for 5hours and after that shudown and sleep for 19h

BEGIN 2022-02-07 06:00:00
END 2023-09-06 11:00:00
ON H5
OFF H19

[mcguirepr89]

Hi, Johannes! Awesome stuff you have going on here! I'm still going through it all, but I think can address and hopefully resolve your crontab issue. I don't quite have enough context to know exactly where this crontab is slated for execution, e.g., is it the pi user's cron only accessible by running crontab -e, or is this the system /etc/crontab to be executed by the user specified?

There are two possible reasons why your cron is not triggering:

1. The user is not indicated in the crontab
2. You've called this bash script with the sh shell, which are not always compatible.

If indeed that cron is in /etc/crontab, then you should adjust it to look like the following and it will work.

30 6-10 * * * pi /home/pi/birdnetpi_github/birdnetpi_github.sh

You'll notice I replaced your call to a shell ("sh") to the pi user. There are two reasons for this:

1. To quote the crontab itself (notice the arrow that I added that points to the relevant part):

   # Example of job definition:
   # .---------------- minute (0 - 59)
   # |  .------------- hour (0 - 23)
   # |  |  .---------- day of month (1 - 31)
   # |  |  |  .------- month (1 - 12) OR jan,feb,mar,apr ...
   # |  |  |  |  .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
   # |  |  |  |  |
   # *  *  *  *  * user-name command to be executed  <<<<<<<------------------This is the important part
   17 *  * * * root    cd / && run-parts --report /etc/cron.hourly
   25 6  * * * root  test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
   

and
2. You originally invoked the sh shell, (which itself is a symbolic link to the dash shell), to run a bash script. The top line of your script already indicates the shell that should execute the commands therein (#!/bin/bash) -- invoking a different shell can lead to problems. If you need to invoke a shell to run the script (because the file itself has not been made executable), then you will want to invoke the script using the shell that is indicated in its "shebang" (the top line), so it would be bash /home/pi/birdnetpi_github/birdnetpi_github.sh in that case.

Let me know if that information helps you get the cron job firing as desired :)

[gitalm]

Dear @mcguirepr89 it is running now and uploading.

Thanks

[mcguirepr89]

You are very welcome!

[gitalm]

IT is running without WiFi and independently powered by solar Power.

Every 15Minutes its Uploading to Github and analyzed by Observablehq.com https://observablehq.com/@gitalm/fetch-data-from-gist-raw-github-url

I just have a problem. I'm synchronicing the folder By_Common_Name but the mp3 files are empty in my Github https://github.com/gitalm/BirdNetPiGitHub/tree/master

[mcguirepr89]

Hi, @gitalm --

In version .11.1 and beyond, there is no longer a "By_Common_Species" directory, only "By_Date".

If you want to re-enable the symbolic links to create the "By_Common_Species" directories, you'll need to alter the extract_new_birdsounds.sh script.

Add the following after line 70:

NEWSPECIES_BY_COMMON="${EXTRACTED}/By_Common_Name/${COMMON_NAME// /_}"

Add these lines after line 88:

    # If a directory does not already exist for the species (by-species),
    # it is created.
    [[ -d "${NEWSPECIES_BY_COMMON}" ]] || mkdir -p "${NEWSPECIES_BY_COMMON}"

And add these lines (around line 130) just above the line that reads done < "${TMPFILE}" so that it looks like the section below:

    # Remove the oldest symbolic links that would make the directory have more
    # than 20 entries.
    if [[ "$(find ${NEWSPECIES_BY_COMMON} | wc -l)" -ge 40 ]];then
      echo "20 ${SPECIES}s, already! Removing the oldest by-species and making a new one"
      cd ${NEWSPECIES_BY_COMMON} || exit 1
      ls -1t . | tail -n +40 | xargs -r rm -vv
      ln -fs "${NEWSPECIES_BYDATE}/${NEWFILE}"\
        "${NEWSPECIES_BY_COMMON}/${NEWFILE}"
      ln -fs "${NEWSPECIES_BYDATE}/${NEWFILE}.png"\
        "${NEWSPECIES_BY_COMMON}/${NEWFILE}.png"
      echo "Success! New extraction for ${COMMON_NAME}"
    else
    # Make symbolic link of the extraction to add to By_Common_Name
      ln -fs "${NEWSPECIES_BYDATE}/${NEWFILE}"\
        "${NEWSPECIES_BY_COMMON}/${NEWFILE}"
      ln -fs "${NEWSPECIES_BYDATE}/${NEWFILE}.png"\
        "${NEWSPECIES_BY_COMMON}/${NEWFILE}.png"
    fi   

done < "${TMPFILE}"

Let me know if you have any questions and I'm happy to help get your setup working again

[mcguirepr89]

I'm so sorry, @gitalm -- I just realized how wrong I was about my comment above -- version 0.11.1 does still have the By_Common_Name directory structure (versions 0.12 and beyond are those that have discontinued the By_Common_Species directory structure).

It seems like we'll need to troubleshoot why your directories were empty on GitHub -- are they empty on the BirdNET-Pi?

[gitalm]

Dear Patrick,

thanks for the help. The problem was the following command

cp -aLru /home/pi/BirdSongs/Extracted/By_Common_Name/* /home/pi/birdnetpi_github/By_Common_Name

I added the L because cp command just copied the symbolic link if it is used recursively within a folder. With the L option it copied the real files associated with the symbolic link.

I also managed to play the files from my GitHub Copy and draw a spectrogram

[mcguirepr89]

Very cool! This could be a cool companion system for a very pared-down BirdNET-Pi that just collects and sends data.:thinking:

[gitalm]

I shut down most services to safe energy with the cron service and it should synchronize the data every 15min.

You need a GitHub Account for GitHub and ObservableHQ, but all data can be forked.

#birdnet GitHub
*/15 6-10 * * * pi /home/pi/birdnetpi_github/birdnetpi_github.sh
#stop services
@reboot root systemctl disable --now caddy.service
@reboot root systemctl disable --now web_terminal.service
@reboot root systemctl disable --now pushed_notifications.service
@reboot root systemctl disable --now livestream.service

[gitalm]

Dear Patrick,

i think the idea from a companion system would be nice. The system via GitHub isn't in concurrence, but would enhance the system.
Benefits:

• data safety
• easy deployment in the internet
• more flexible installations
  • due to internet connectivity
  • due to power supply

Your default installation is just magnificent, deploy a RPi-Image and run a script to get a full featured AI-Bird Analysis system within your intranet.

To get both possibilities i think two parts have to be added

• GitHub and the mechanism to safe data there (synchronization is built in)
• maybe a way to reduce the services, by stopping them

I also tried to load the d3.js library directly in GitHub pages and succeed in loading the csv file, but development was to tricky for me and observablehq is an very easy way to copy snippets and get a full analysis of the data from BirdNet Pi.

Best wishes Johannes

PS: Dear Patrick i don't think this solutions is competitive to your wonderful RPi based solution, probably somehow alternative and with some additional feature due to an different focus.

[mcguirepr89]

Once I have the chance, I will definitely tinker with this.

PS: Dear Patrick i don't think this solutions is competitive to your wonderful RPi based solution, probably somehow alternative and with some additional feature due to an different focus.

I appreciate your consideration, but I truly am in no competition for these systems. I just want to be sure that folks like me with a love of birds can get their hands on this awesome technology with as little trouble as possible. Indeed, if you made a system I liked better than BirdNET-Pi, that is the system I would use!

Please excuse my slowness in getting more familiar with this neat feature/setup -- once I've had an opportunity to tinker with it myself, I will have a better idea of how it might be included as a BirdNET-Pi something-or-another.

[mcguirepr89]

@gitalm !!! How are you???

I wanted to let you know that your project sparked an idea that I think I'll be putting into the new version I'm working on for the new model, and then will work on retro-fitting it for this current BirdNET-Pi version.

The new BirdNET-Pi is structured a bit differently: everything will be inside the git directory for the project (including the service files and all data that is collected). If I initialize a "Segments" directory (which is pretty much the new BirdNET-Pi's version of the current "By_Date" directory containing all extraction data) in the repo, GitHub can be the "always online" solution for these timed solar projects. I can also make a very rudamentary script to run the necessary git config

The process would be a little weird to setup for folks not comfortable managing SSH keys:

• Sign up for a GitHub account if you don't have one already
• Fork BirdNET-Pi
• Edit the .gitignore file in the fork to remove the following for uploading to GitHub:
  • Segments <-- This is tantamount to the "By_Date" directory containing all the birdsong extractions, so removing this will allow you to store a copy of all your extractions on a GitHub server
  • birds.db <-- Removing this will allow you to store a copy of your SQLite database on a GitHub server
• Run the installer from the fork so that the installation's upstream is the fork
• Go to the installation's Web GUI to generate an SSH Key pair and copy the id_ed25519.pub and input the git username and email
  • (A script would run here that would do the proper git config user.name "$git_user, git config user.email "$git_email", git remote set-url origin git@github.com:$git_user/BirdNET-Pi.git
• Paste id_ed25519.pub into the GitHub settings SSH authorized keys

This way, there can be a cron job (like your project) which simply runs git -C /home/pi/BirdNET-Pi commit -a && git -C /home/pi/BirdNET-Pi push to have all of the "Segments" (birdsong data) and birds.db database available online even when the BirdNET-Pi is powered off!

I don't know yet if that is exactly how it would go, but I think it is a really fun idea! Thank you for it!

[gitalm]

Dear @mcguirepr89 , this sounds wonderful. I was a bit busy the last days, but if i can help testing and so on let me know.
I think it is very important that this is not something totally new, because BirdNET-Pi is wonderful and your small hack will enhance it for especially to put it outdoor. I guess it will be the most forked GitHub🦉🧠🎙️, hopefully more clones than dolly 🐑🐑🐑🐑🐑🐑🐑

[mcguirepr89]

I guess it will be the most forked GitHub 🦉🧠🎙️,hopefully more clones than dolly 🐑🐑🐑🐑🐑🐑🐑

hahaha 😆

I will let you know when I have something that you can test! THANK YOU FOR YOUR SPONSORSHIP!!!!

[mcguirepr89]

Also, your solar setup will likely benefit from running things on the Raspberry Pi 0 W 2, which I have decided I will officially support since testing has proven so promising.

The power requirements for the Raspberry Pi 0W2 won't be comparable to the RPi0W1, but should be less than the power needs of the Raspberry Pi 3B+. Depending on whether or not you can get your hands on a RPi0W2, I would be really interested to see whether the lower power constraints free up some time on your solar system :)

I put together a brief set of instructions for that hardware here

My best!