DietPi-Software | Sonarr: Migrate to v3 #4292

MichaIng · 2021-04-21T13:35:54Z

Status: Ready

Commit list/description:

DietPi-Software | Sonarr: Migrate to v3
DietPi-Software | Mono: Remove obsolete "-O=-aot" from Mono calls, which was a workaround that has been fixed in the meantime: DietPi-Software | Mono: Current version fails on ARM #2219
DietPi-Software | Complement and align Arr service hardenings

+ DietPi-Software | Sonarr: Migrate to v3 + DietPi-Software | Mono: Remove obsolete "-O=-aot" from Mono calls, which was a workaround that has been fixed in the meantime: #2219 + DietPi-Software | Complement and align Arr service hardenings

+ DietPi-Software | Arr services cannot use PrivateTmp=true, else DietPi-Arr_to_RAM does not work + DietPi-Software | Sonarr: Pre-configure DEB package and tune install and uninstall

+ DietPi-Patches | Inform user about possible Sonarr v2 => v3 upgrade

+ DietPi-Arr_to_RAM | Add support for Sonarr v3 and Radarr v3 + DietPi-Arr_to_RAM | On first link to RAM, create a script in the programs data dir to update the linked database backups, as v3 do not support arguments to pass to custom scripts anymore and do not allow to start scripts from within /boot.

+ DietPi-Arr_to_RAM | Satisfy shellcheck

+ CHANGELOG | DietPi-Arr_to_RAM: Support for Sonarr v3 and Radarr v3 has been added. On first link to RAM, a script "/mnt/dietpi_userdata/(sonarr|radarr|lidarr)/dietpi-arr_to_RAM.sh" is created, which allows updating the linked database backups via the programs "Custom Script" feature. Since v3, it is not possible anymore to pass arguments to custom script or call scripts inside the /boot directory at all, which broke the previous "/boot/dietpi/misc/dietpi-arr_to_RAM 2 (sonarr|radarr|lidarr)" calls in two ways. + CHANGELOG | Sonarr: Support for and migration to v3 has been implemented. Existing installs won't be migrated automatically, run "dietpi-software reinstall 144" to upgrade your Sonarr to v3. On DietPi update, Sonarr v2 users will receive a related notification.

+ DietPi-Software | Sonarr: Install mediainfo, which is suggested by the package but no strict dependency. Align with Radarr and Lidarr. + DietPi-Software | Radarr: Remove Radarr v2 PID file and database backups from DietPi-Arr_to_RAM, which can be left overs

+ DietPi-Arr_to_RAM | Use safer approach to create database backups: The -shm and -wal files do sometimes not exist, e.g. when the connection is closed temporarily. In theory the should never exist when the program is not running, but in practice it does at least with Sonarr and Lidarr, which obviously do not close connections gracefully or use certain SQLite flags to keep those files intentionally. However, havoc can happen when those files do not exist while the database is linked to RAM or stored back to disk: In the first case, if the program then crashes, up-to-date -shm + -wal files will be mixed with a backup database file, in the second case, the -shm and -wal symlinks will stay on disk, breaking program start as the RAM dir is removed. Especially in combination with the automatic backup restoring, mixing of old and new files can happen, corrupting the database. At least when linking to RAM and for backup creation, SQLite is used now to read and store the database to the backup location, which includes -shm and -wal, so that a single consistent database file is present for the program to start with.

+ DietPi-Arr_to_RAM | Simplify and harden script

+ DietPi-Arr_to_RAM | Assure that program user has write permissions to new database backups

+ DietPi-Arr_to_RAM | Fixes

+ DietPi-Patches | Remove obsolete DietPi-Arr_to_RAM backup files, as now "sqlite3 db .save" is used to store a single db file as backup

+ DietPi-Software | Skip DietPi-Arr_to_RAM backup -shm/-wal removal which is now done during DietPi update

Tooloco · 2021-04-29T17:46:15Z

Hello, I dont know how to use github very well. I just wanted to point out that there seems to be an issue in this script /dietpi/misc/dietpi-arr_to_RAM at line 79. When parsing arguments the if statements checks for sonarr instead of radarr. I also wanted to point out that in the last update (7.1.2) the scripts created by the dietpi-arr_to_ram 2 command (when migrating sonarr or radarr) have a typo that wont let the command run. I believe I've explained myself if not get back to me.

Joulinar · 2021-04-29T17:48:34Z

Yes we know and already fixed it.

#4315

Tooloco · 2021-04-29T17:49:16Z

Ok, is the patch up right now?

edit: thanks for the fast answer !

Joulinar · 2021-04-29T17:50:27Z

If you already created the script, it would need to be changed manually

Edit
At least the typo we correct

Tooloco · 2021-04-29T17:56:41Z

Ok, I'll do those manually then. What about the other problem on the argument check. Is that something that is known ? here: https://github.com/MichaIng/DietPi/blob/master/dietpi/misc/dietpi-arr_to_RAM
line 79

Joulinar · 2021-04-29T17:58:25Z

Not yet 😃

DietPi/dietpi/misc/dietpi-arr_to_RAM

Line 79 in 86c5e37

if [[ ${INPUT_PROG:-sonarr} == 'sonarr' ]]

MichaIng · 2021-04-29T17:58:29Z

True, another typo.

Joulinar · 2021-04-29T17:59:02Z

@MichaIng
Are you going to do the change?

MichaIng · 2021-04-29T18:00:26Z

Done: 4b171a1

Tooloco · 2021-04-29T18:03:45Z

Wow, you sure work fast guys! I cannot be happier with dietpi!

MichaIng · 2021-05-01T18:07:54Z

What happens when you call it manually?

sudo -u sonarr /mnt/dietpi_userdata/sonarr/dietpi-arr_to_RAM.sh

gorby-pranata · 2021-05-02T01:45:20Z

2021-05-02 08:43:53 | DietPi-Arr_to_RAM: Radarr detected
2021-05-02 08:43:53 | DietPi-Arr_to_RAM: Updating Radarr database backup...
Error: attempt to write a readonly database
2021-05-02 08:43:53 | DietPi-Arr_to_RAM: [ERROR] Updating Radarr database backup failed.
2021-05-02 08:43:53 | DietPi-Arr_to_RAM: Sonarr detected
2021-05-02 08:43:53 | DietPi-Arr_to_RAM: Updating Sonarr database backup...
2021-05-02 08:43:53 | DietPi-Arr_to_RAM: Updated Sonarr database backup.
2021-05-02 08:43:53 | DietPi-Arr_to_RAM: [ERROR] An issue has occurred. Please check the above output for details.

This are the results.

Joulinar · 2021-05-02T06:41:29Z

Just to be sure, you don't have Radarr installed? Correct?

gorby-pranata · 2021-05-02T08:26:27Z

Just to be sure, you don't have Radarr installed? Correct?

i have both sonarr and radarr installed, sorry if i didn't address this earlier. I thought the fix would apply to both radarr and sonarr

Joulinar · 2021-05-02T08:29:32Z

There is one more issue for detecting Radarr correctly. It would require correction on line 79

DietPi/dietpi/misc/dietpi-arr_to_RAM

Line 79 in 86c5e37

if [[ ${INPUT_PROG:-sonarr} == 'sonarr' ]]

gorby-pranata · 2021-05-02T09:05:11Z

There is one more issue for detecting Radarr correctly. It would require correction on line 79

DietPi/dietpi/misc/dietpi-arr_to_RAM

Line 79 in 86c5e37

if [[ ${INPUT_PROG:-sonarr} == 'sonarr' ]]

i'm a newbie with coding and this stuff, so i'm not sure i can figure a way how to correct that line. Would there be a patch that i can just run in the command line that would fixed both radarr and sonarr with dietpi to ram script?

MichaIng · 2021-05-02T09:26:57Z

Do you have MOTD in the DietPi banner enabled? If so, can you paste the output of:

cat /run/dietpi/.dietpi_motd

Tooloco · 2021-05-02T09:53:15Z

Just adding to this that until I manually ran dietpi-banner the fix did not happen. So it might help running it.

MichaIng · 2021-05-02T10:11:41Z

until I manually ran dietpi-banner

But it runs on any login, doesn't it?

gorby-pranata · 2021-05-02T10:29:31Z

Do you have MOTD in the DietPi banner enabled? If so, can you paste the output of:
cat /run/dietpi/.dietpi_motd

This is the output

MichaIng · 2021-05-02T10:31:32Z

That's a very old MOTD which should have been replaced long ago 🤔. Can you please paste the following:

journalctl --no-pager -u cron | grep daily

gorby-pranata · 2021-05-02T10:35:10Z

That's a very old MOTD which should have been replaced long ago 🤔. Can you please paste the following:
journalctl --no-pager -u cron | grep daily

Okay... I don't know how it got wrong.. this is the output for the journalctl command

root@DietPi:~# journalctl --no-pager -u cron | grep daily
Apr 29 01:25:01 DietPi CRON[13999]: (root) CMD (test -x /usr/sbin/anacron || { cd / && run-parts --report /etc/cron.daily; })
Apr 30 01:25:02 DietPi CRON[19250]: (root) CMD (test -x /usr/sbin/anacron || { cd / && run-parts --report /etc/cron.daily; })
May 01 01:25:01 DietPi CRON[22570]: (root) CMD (test -x /usr/sbin/anacron || { cd / && run-parts --report /etc/cron.daily; })
May 02 01:25:01 DietPi CRON[12018]: (root) CMD (test -x /usr/sbin/anacron || { cd / && run-parts --report /etc/cron.daily; })

MichaIng · 2021-05-02T10:41:57Z

I think I found the issue. The following file does not exist, does it?

ls -l /boot/dietpi/.dietpi-banner

The MOTD is only updated daily, if MOTD is actually enabled in the banner: https://github.com/MichaIng/DietPi/blob/master/rootfs/etc/cron.daily/dietpi#L38-L43
To check this, we however check for the config file (the one above), which is only created if one manually changes the banner settings. Since the MOTD is enabled by default (when no config file exists), it needs to be updated as well in that case.

And I was wondering why still so many users suffer from issues that should be fixed already 😄.
Fixed: a513669
Changelog: 0d20976

In you case:

rm /run/dietpi/.dietpi_motd
curl -sSfL https://dietpi.com/motd -o /run/dietpi/.dietpi_motd
dietpi-banner 1

gorby-pranata · 2021-05-02T10:46:23Z

I think I found the issue. The following file does not exist, does it?
ls -l /boot/dietpi/.dietpi-banner
The MOTD is only updated daily, if MOTD is actually enabled in the banner: https://github.com/MichaIng/DietPi/blob/master/rootfs/etc/cron.daily/dietpi#L38-L43
To check this, we however check for the config file (the one above), which is only created if one manually changes the banner settings. Since the MOTD is enabled by default (when no config file exists), it needs to be updated as well in that case.

And I was wondering why still so many users suffer from issues that should be fixed already 😄.

okay so should i just wait till tomorrow and use the fix the MOTD offer or am i missing something? again really appreciate your time and effort in answering my questions😁

MichaIng · 2021-05-02T10:49:11Z

Run the commands I edited into my post 🙂.

gorby-pranata · 2021-05-02T10:51:40Z

Run the commands I edited into my post 🙂.

Awesome! the fix works wonderfully! thank you so much for the time and effort! You're amazing 😁

MichaIng · 2021-05-02T11:00:38Z

Thanks for your kind feedback and responsiveness here as well. Great that we finally found the reason for users seeing outdated banner messages (and not receiving fixes we hack inside).

I hope I find time for next release to add a dedicated transparent live-patch feature that allows to interactively apply such fixes, like "A patch is available for issue X, run dietpi-live-patches to apply it to your system" or so. So we can do fixes for issues that can be safely resolved with a single sed between releases, but transparently instead of the current sneaky way 😅.

gorby-pranata · 2021-05-02T11:06:30Z

Thanks for your kind feedback and responsiveness here as well

Very happy to help 😊!

Tooloco · 2021-05-02T11:21:45Z

until I manually ran dietpi-banner

But it runs on any login, doesn't it?

The banner does show whenever I ssh into it but I manually had to nano in to the arr to ram script to fix it.

Joulinar · 2021-05-02T11:42:27Z

@Tooloco
do you have MOTD activated?

MichaIng · 2021-05-02T11:54:33Z

Same reason, I guess: #4292 (comment)
The MOTD was not updated as expected, so the fix was not applied as expected 😉.

Tooloco · 2021-05-02T12:01:59Z

@Tooloco
do you have MOTD activated?

Yes, and I had manually configured the banner, adding all options but vpn status.

MichaIng · 2021-05-02T12:08:52Z

Ah, you mean the when you reported the issue here? #4292 (comment)
The fix has been applied after (and thanks to!) your report 🙂.

Tooloco · 2021-05-02T12:09:25Z

Same reason, I guess: #4292 (comment)
The MOTD was not updated as expected, so the fix was not applied as expected 😉.

dietpi@DietPi3:~$ cat /run/dietpi/.dietpi_motd
# Live-patches
if [[ $G_DIETPI_VERSION_SUB == 1 ]]
then
        if [[ -w '/boot/dietpi/misc/dietpi-arr_to_RAM' ]]
        then
                grep -q 'dietpi-arr_to_RAN' /boot/dietpi/misc/dietpi-arr_to_RAM && sed -i 's/dietpi-arr_to_R
AN/dietpi-arr_to_RAM/' /boot/dietpi/misc/dietpi-arr_to_RAM
                [[ $(sed -n 79p /boot/dietpi/misc/dietpi-arr_to_RAM) == *sonarr* ]] && sed -i "79c\\\tif \[\
[ \${INPUT_PROG:-radarr} == 'radarr' \]\]" /boot/dietpi/misc/dietpi-arr_to_RAM
        fi
fi
# MOTD
motd='Did you know that you can run "dietpi-banner" to change its content?'

Tooloco · 2021-05-02T12:11:48Z

Ah, you mean the when you reported the issue here? #4292 (comment)
The fix has been applied after (and thanks to!) your report 🙂.

OK just making sure everything is OK its running smoothly now, I've added the script to sonarr and radarr and it's working flawlessly.

MichaIng · 2021-05-02T12:14:47Z

Perfect, many thanks for validating.

v7.1

9ffa407

+ DietPi-Software | Sonarr: Migrate to v3 + DietPi-Software | Mono: Remove obsolete "-O=-aot" from Mono calls, which was a workaround that has been fixed in the meantime: #2219 + DietPi-Software | Complement and align Arr service hardenings

MichaIng added the Enhancement 💨 label Apr 21, 2021

MichaIng added this to the v7.1 milestone Apr 21, 2021

MichaIng linked an issue Apr 21, 2021 that may be closed by this pull request

DietPi-Software | Sonarr: Migrate from v2 to v3 #4241

Closed

MichaIng added 8 commits April 21, 2021 16:49

v7.1

210cd3d

+ DietPi-Software | Arr services cannot use PrivateTmp=true, else DietPi-Arr_to_RAM does not work + DietPi-Software | Sonarr: Pre-configure DEB package and tune install and uninstall

v7.1

cc564fa

+ DietPi-Patches | Inform user about possible Sonarr v2 => v3 upgrade

v7.1

223e6c4

+ DietPi-Arr_to_RAM | Satisfy shellcheck

v7.1

34519da

+ DietPi-Software | Sonarr: Install mediainfo, which is suggested by the package but no strict dependency. Align with Radarr and Lidarr. + DietPi-Software | Radarr: Remove Radarr v2 PID file and database backups from DietPi-Arr_to_RAM, which can be left overs

v7.1

86eec61

+ DietPi-Arr_to_RAM | Simplify and harden script

MichaIng requested review from ravenclaw900 and Joulinar April 21, 2021 19:07

MichaIng added 4 commits April 21, 2021 21:18

v7.1

98dc812

+ DietPi-Arr_to_RAM | Assure that program user has write permissions to new database backups

v7.1

8695f81

+ DietPi-Arr_to_RAM | Fixes

v7.1

083db72

+ DietPi-Patches | Remove obsolete DietPi-Arr_to_RAM backup files, as now "sqlite3 db .save" is used to store a single db file as backup

v7.1

6ee9949

+ DietPi-Software | Skip DietPi-Arr_to_RAM backup -shm/-wal removal which is now done during DietPi update

MichaIng merged commit 4778fe7 into dev Apr 21, 2021

MichaIng deleted the sonarr branch April 21, 2021 20:20

Joulinar mentioned this pull request May 5, 2021

DietPi-Arr_to_RAM | Error in update script #4348

Closed

This was referenced May 22, 2021

Beta v7.2.0 #4406

Merged

Open Beta v7.2 | Please help testing and hardening the upcoming release #4409

Closed

Release v7.2 #4422

Merged

DietPi-Software | Sonarr: Migrate to v3 #4292

DietPi-Software | Sonarr: Migrate to v3 #4292

Conversation

MichaIng commented Apr 21, 2021 • edited Loading

Tooloco commented Apr 29, 2021

Joulinar commented Apr 29, 2021 • edited Loading

Tooloco commented Apr 29, 2021 • edited Loading

Joulinar commented Apr 29, 2021 • edited Loading

Tooloco commented Apr 29, 2021

Joulinar commented Apr 29, 2021

MichaIng commented Apr 29, 2021

Joulinar commented Apr 29, 2021

MichaIng commented Apr 29, 2021

Tooloco commented Apr 29, 2021

MichaIng commented May 1, 2021

gorby-pranata commented May 2, 2021

Joulinar commented May 2, 2021

gorby-pranata commented May 2, 2021

Joulinar commented May 2, 2021

gorby-pranata commented May 2, 2021 • edited Loading

MichaIng commented May 2, 2021 • edited Loading

Tooloco commented May 2, 2021

MichaIng commented May 2, 2021

gorby-pranata commented May 2, 2021 • edited Loading

MichaIng commented May 2, 2021 • edited Loading

gorby-pranata commented May 2, 2021 • edited Loading

MichaIng commented May 2, 2021 • edited Loading

gorby-pranata commented May 2, 2021

MichaIng commented May 2, 2021

gorby-pranata commented May 2, 2021

MichaIng commented May 2, 2021

gorby-pranata commented May 2, 2021

Tooloco commented May 2, 2021

Joulinar commented May 2, 2021

MichaIng commented May 2, 2021 • edited Loading

Tooloco commented May 2, 2021

MichaIng commented May 2, 2021

Tooloco commented May 2, 2021 • edited Loading

Tooloco commented May 2, 2021

MichaIng commented May 2, 2021

MichaIng commented Apr 21, 2021 •

edited

Loading

Joulinar commented Apr 29, 2021 •

edited

Loading

Tooloco commented Apr 29, 2021 •

edited

Loading

Joulinar commented Apr 29, 2021 •

edited

Loading

gorby-pranata commented May 2, 2021 •

edited

Loading

MichaIng commented May 2, 2021 •

edited

Loading

gorby-pranata commented May 2, 2021 •

edited

Loading

MichaIng commented May 2, 2021 •

edited

Loading

gorby-pranata commented May 2, 2021 •

edited

Loading

MichaIng commented May 2, 2021 •

edited

Loading

MichaIng commented May 2, 2021 •

edited

Loading

Tooloco commented May 2, 2021 •

edited

Loading