Manifest v2

[Salamandar]

Problem

• Description of why you made this PR

Solution

• And how do you fix that problem

PR Status

• Code finished and ready to be reviewed/tested
• The fix/enhancement were manually tested (if applicable)

Automatic tests

Automatic tests can be triggered on https://ci-apps-dev.yunohost.org/ after creating the PR, by commenting "!testme", "!gogogadgetoci" or "By the power of systemd, I invoke The Great App CI to test this Pull Request!". (N.B. : for this to work you need to be a member of the Yunohost-Apps organization)

[Salamandar]

!testme

[yunohost-bot]

Alrighty!

[zamentur]

!bookwormtestme

[yunohost-bot]

📚

[zamentur]

!bookwormtestme

[yunohost-bot]

📚 🪱

[fflorent]

Sorry, misclicked on approval

[fflorent]

See above

[Salamandar]

The app as is doesn't seem to work. Here is a log of a failed backup https://paste.yunohost.org/raw/okesisubiz

Found the issue : python3 -m venv --upgrade does not create the activate script (why ???).

Issue is coming from this line https://github.com/python/cpython/blob/76dc1bf5be3b360f7d8c476accb0d8d53c8dfb08/Lib/venv/__init__.py#L77

EDIT: fixed by 178a24b, i just call borg directly in the script instead of using activate.

@Limezy thank you for having accepted to test this branch :) Could you please try to upgrade to the latest commit and check if it’s better ?

[Limezy]

I have upgraded and launched a backup, it takes crazy long, as usual !
I'll give a feedback once done, hopefully

[alexAubin]

(Smol poke in case you have feedback now)

[Limezy]

Still waiting for my server... as of now my /var/log/borg/xxx.log is empty and /var/log/borg/xxx.err says still waiting...
And the Yunohost-api is locked as if a Yunohost operation was pending

[Limezy]

I have restarted my server, which relaunched a backup.
I think the service should be disabled by default ?

[alexAubin]

That's a known issue : #92

[Limezy]

So, here is my situation, it looks like a part of my backups worked, a part failed.
content of /var/log/borg/240515_0605.log : https://paste.yunohost.org/raw/holakupapu
output of yunohost log share 20240515-061238-backup_create : https://paste.yunohost.org/raw/afokajutol
content of /var/log/borg/240515_0605.err : https://paste.yunohost.org/raw/wovizigabo
last lines of journalctl for the borg service (but I think nothing interesting) : https://paste.yunohost.org/raw/ekonetohub

I must also mention that I get this weird warning when updating the config panel :

[Limezy]

@Salamandar thanks I confirm that the config panel changes are now made without any warnings.
Now launching again a backup.

[Limezy]

@alexAubin, looking at my logs, any clue about what's going wrong with the backup of MinIO and Outline (both apps for which I am / was the packager 😰 maybe I did something wrong ?)

Moreover it says in the doc that "The future default backup method integrated in YunoHost will be based on this software.". Is this very far future ? Would be absolutely awesome to have a nice Yunohost UX layer on top of all these geeky private keys and complex setups... And certainly a real added value of the project.

I decided to take seriously my backup pipeline after the OVH fire back in... March 21. Only tackling it now... Backup is probably a big area where self-hosting need to improve.

[Limezy]

I have discovered that when I launch yunohost app setting borg passphrase it outputs no value.
Whereas yunohost app setting borg repository is working well.
Is that maybe one of the clues of our problem ?

[Salamandar]

@Limezy might be… at least yeah there's a bug in the install script for sure, but not in the upgrade script.

[Limezy]

Ok let me upgrade and try again even though it's probably another issue !

[Limezy]

I have tried to reinstall from scratch, using this branch.
In fact I have the feeling that Borg isn't installed !
Maybe it worked before because I did an upgrade only ?

EDIT: Sorry I found it here : /var/www/borg/venv/bin/borg

[alexAubin]

Smol poke, is it working now ? 👀

[Limezy]

I have excluded dex and outline and everything is working. But the problem still remains for these two apps. What I'm not clear is whether the problem is at borg_ynh level, these apps backup script level, or Yunohost core level ? The above shared logs remain relevant.
Thanks for your help I'd really like this to be working !

[Limezy]

@alexAubin by chance would you have any hint about what's blocking Dex and Outline from being automatically backed up ?

[alexAubin]

It's not clear to me what's the corresponding log x_x

[Limezy]

OK, sorry !

Basically, the auto backup + sending to Borgwarehouse repo is working for most of the apps despite throwing a Calling the custom backup method '{method}'...' error.
For two other apps (Dex and Outline), it's completely failing with an additional Moulinette error as per the log https://paste.yunohost.org/raw/wovizigabo

I'm thinking it's probably not a borgwarehouse problem since it's able to perfectly receive some other backups for other apps. Is it a borg_ynh app bug ? A dex_ynh / outline_ynh not well implemented backup script error ? A Yunohost core (moulinette) bug ?

[alexAubin]

So dex seems to be failing because, i'm guessing, install_dir is not defined, which is weird and inconsistent, I'm not sure how this happened ... You can confirm this with yunohost app setting get dex install_dir (which i guess will be empty) and fix with yunohost app setting set dex install_dir -v /var/www/dex

I'm confused about outline, there's definitely a bug in the core but at the same time i think you're hitting this bug maybe because of unusual mount configuration ? Do you have symlinks / bind mounds maybe in your outline data/install folders or something ?

Anyway these two things are unrelated to borg itself

[Limezy]

Many thanks for your reply. Shall we continue this discussion on the forum ? In the app dev chat ? Meanwhile I'll investigate based on this first feedback.

[Limezy]

@alexAubin I guess the intended command was yunohost app setting dex install_dir instead ?
If yes, the return is /var/www/dex so it doesn't look like the cause of our problem 🫤

[alexAubin]

Then we need to look at the full log of yunohost backup create --apps dex --debug

[Limezy]

Indeed you are right... I had a stupid source path unknown error. After an update of Dex the command is running. Now retrying a full server backup with Borg_ynh

[alexAubin]

Soooo if there's no more issues, are we good to go ?

[alexAubin]

The ljf god gave us his blessing !