Exclude specific folders from auto-upload discovery

[vincentmathis]

Is your feature request related to a problem? Please describe.

I want to use the auto-upload feature and sometimes configure some custom folders. The problem here is that user potentially have a large number of folders containing a single album cover. In my case this means that opening the auto-upload page in the app takes 10+ seconds and sometimes even crashing and then shows 3000+ folders with a single image. I can't find the folders i actually want to manage unless i scroll for minutes.

Describe the solution you'd like

It would be nice to have a setting to exclude a path from the auto-upload discovery where one could pick to exclude everything in internal/music from auto-upload.

Describe alternatives you've considered

I do not know a way how to hide these folders except manually select "hide folder" for thousands of folders.

[AndyScherzinger]

Hi @vincentmathis,
a heuristic for the items only list has been discussed and we agreed that we wouldn't add a possibility to manage that yourself but rather a set of rules that would prevent such entries.

So can you elaborate on these folder, storage path is rather to specific but folder names and cover art names, number of further images in that folder, possible names so we could implement a "smart" check to auto-exclude such folders :)

[vincentmathis]

In my case they are all called Music/{artist}/{album}/cover.jpg because I used the same tool for all my music.

A possible way might be to check if a folder contains bunch of audio files (mp3, wav etc.) and one image file. That would usually indicate an music album with cover image. I just don't know how one would deal with false positives.

[netphantm]

This would be very useful, I also have a bunch of them, because the player I use downloads them automatically. I would disable that, but it's nice to see the cover when the album/piece is playing. They are created and placed under Music/{artist}/{album}/covers/[back|booklet(1..x)|cd|front].jpg. Ideally I would want to be able to blacklist the whole Music folder from autodiscovery, I think it would be the simplest thing.

[AndyScherzinger]

@jancborchardt @tobiasKaminsky what do you think? With #7042 (shipping with the next release) we will ignore/exclude folder with a single image (no other images) that is named cover|folder.jpg.

@netphantm do these folder contain just a single image or can there be all/some files based on your expression [back|booklet(1..x)|cd|front].jpg. Also @jancborchardt @tobiasKaminsky I'd be fine to also ignore the music folder (since auto discovery is for images and if someone puts them under "music" then the need to use the manual/custom folder mechanisms but loose auto-discovery (which seem s totally fine to me).

[AndyScherzinger]

@vincentmathis see comment above,the upcoming release will ignore folders with a single image named cover|folder.jpg (if you put in any other image we will detect it again (!)

[EtlamGit]

I would prefer to ignore images in complete Music folder.
My audio tools generate also images called %artist%_-_%album%.jpg and several other variants.

[netphantm]

the folder inside the artist/album/ is named covers and contains these files on my phone: (back|booklet\d.+|cd|front).jpg
Why not just create an rsync blacklist with folders which should be excluded (including subfolders)? Seems more intuitive, like w/ the --exclude-from switch in rsync.

[netphantm]

I would prefer to ignore images in complete Music folder.
My audio tools generate also images called %artist%_-_%album%.jpg and several other variants.

right, also an alternative.
But in my case I would like to be able to ignore the whole folder, since I don't sync mp3s over WiFi anyway.
Also, some folders contain a hidden file called smth like .Thumbs.db, (they come from some stupid image file viewer I suppose). My mp3 collection is pretty old, and I used a lot of different players and image viewers over the time (I too was a windoze user some 15 years ago, before linux :-/ ).
When I copy new music from it to the phone, I just select a bunch of artists/disks folders and don't want to have to look into every one to clean up (or can't, since some are still on CDs/DVDs)..

[vincentmathis]

Very nice. That makes the auto-upload usable for me. Thanks for implementing this.

Just like netphantm said I think a white/blacklist for directories could be useful. My gallery app has this and I love it because I can hide entire Folders of images that I don't want to see or add Folders that didn't automatically show up. Extremely useful!

Any reason for not adding this? Maybe confusing "non power users" with extra settings?

Still, this is a major first step for people who have a lot of albums of music on their sdcard.

[AndyScherzinger]

Like you mentioned, the reason is not adding extra settings that might confuse most users but rather find a nice set of fixed rules to improve this automatically for everybody.

And yes, reading through the newest comments here I think it be good to just ignore the music main folder completely.

Folders not auto detected can be added manually by the way (see 3 dot main menu on the top right, called set up custom folder).

[EtlamGit]

And yes, reading through the newest comments here I think it be good to just ignore the music main folder completely.

There are other standard folders on Android phones that could have the same problem: Users put their mp3 collection into Music, their VHS video collection into Video and their favorite photos into Pictures. All of these is unlikely to be intended to be synced into a cloud (as it is copied manually onto the phone).
So you may consider thinking about solutions for all of these 3 folders.
(My favorite would be to ignore all 3 of them.)

[AndyScherzinger]

Ignoring all 3 is highly unlikely at least in my opinion. Pictures is a Android standard folder for images in contrast to DCIM for camera apps. So apps like Twitter, Facebook, etc. put explicitly downloaded images into a subfolder of pictures thus most people would rather want this to be not ignored.

You can always hide folders via the auto upload folder item on the screen.

If you put a '. nomedia' in the respective folder, so Android won't index the files in there and we won't detect and thus ignore it. So that has always been solved via Android standard mechanism ;)

[netphantm]

If you put a '. nomedia' in the respective folder, so Android won't index the files in there and we won't detect and thus ignore it. So that has always been solved via Android standard mechanism ;)

won't this also ignore the mp3s when they should be scanned by music apps?

You can always hide folders via the auto upload folder item on the screen.

I have about 50 folders and subfolders of mp3s, which may contain covers, so it'll be a PITA to manually hide them all after being auto discovered in "Auto upload"..

[AndyScherzinger]

That's true, while the initial suggestion of just ignoring the music folder would still work and is something I'd vote to do.

The thing where I disagree is to completely ignore the Pictures folder since it is used by non-camera apps for image storage.

[cincodenada]

If you put a '. nomedia' in the respective folder, so Android won't index the files in there and we won't detect and thus ignore it. So that has always been solved via Android standard mechanism ;)

won't this also ignore the mp3s when they should be scanned by music apps?

It will indeed, which is why I don't have a .nomedia in my music folder - because that's the whole point, is to...have media. But I care about the music files, not the image files. If I put a .nomedia in my music folder, then my music files don't show up in my music apps.

I'm not sure if it's actually set as a formal standard anywhere, but there's a defacto standard of supporting a similar .noimage file to exclude just images, while allowing indexing of audio files. This would solve the problem here as well, and there are already issues on the NextCloud Web client to support these - I think it's a good solution, as it doesn't require any additional configuration in the app interface, just in the scan logic.

Excluding certain directories (and thus their children) from the scan as a config in the app (as this issue originally suggested) would work just fine for me too, I've seen that in a few gallery-type apps I've used.

[AndyScherzinger]

it doesn't require any additional configuration in the app interface, just in the scan logic.

Problem is the scan logic is part of Android as in operating system while, yes we could check the folders content for such a file which wouldn't be too hard with the latest change to ignore certain folders based on their content 👍 So support a .noimage check is possible while I doubt that people would put this into every album folder (since the noimage check won't be working in a arbitrary way :/

[oligneisti]

I am copying my MP3 collection to my phone. Nextcloud has discovered all these folders that have a couple of images each. Also importing my rip of the DVD boxset if Scrubs. Each episode has a few pictures from the show. Each episode is a new folder for Nextcloud auto-upload.

[Bun-Bun]

Hi @vincentmathis, a heuristic for the items only list has been discussed and we agreed that we wouldn't add a possibility to manage that yourself but rather a set of rules that would prevent such entries.

So can you elaborate on these folder, storage path is rather to specific but folder names and cover art names, number of further images in that folder, possible names so we could implement a "smart" check to auto-exclude such folders :)

I believe that decision is short sighted in this case. Taking the onus on yourselves to make the heuristics for every use case would be a) unobtainable b) impossible to maintain.

Example I find myself wanting to ignore sub folders in my camera apps folder. The structure is this PHOTOGRAPHY_PRO\Burst{abillionfolders}. My camera app creates a new folder for every burst set of photos. Over the course of a couple hours I generated 200 such folders and over 3000 files. The scanning from Nextcloud at a time of heavy i/o from the camera app (it is bursting 20-30 images per second) is causing performance issues with my phone and for next cloud to get into some weird loop and queue up 20,000 uploads. I do not want to upload every single one of these burst photos as the point is to go back and select the ones I do want to keep which then go to the main PHOTOGRAPHY_PRO folder. I simply want to ignore all sub folders of the PHOTOGRAPHY_PRO or Burst folder.

it doesn't require any additional configuration in the app interface, just in the scan logic.

Problem is the scan logic is part of Android as in operating system while, yes we could check the folders content for such a file which wouldn't be too hard with the latest change to ignore certain folders based on their content 👍 So support a .noimage check is possible while I doubt that people would put this into every album folder (since the noimage check won't be working in a arbitrary way :/

.nomedia is not a suitable solution as the folder in question I do want to be available to photo and other apps on the device. I simply do not want the performance issues and consumption of bandwidth/disk space by Nextcloud handling all these many 100s of folders being created daily.

[Starzend]

For inspiration, I host navidrome and the dev included the possibility to add a .ndignore file to folders to be excluded from the scan

[joshtrichards]

Similar: #3239