osxphotos script on Synology

[jeremieberrebi]

Hello,

I have a backup of my Photo Library on my Synology DS1522+ . I would like to export all the photos directly from the Synology to the Synology.

Do you know if someone already done that? What's the best way to install the script on it?

Thanks!

[RhetTbull]

Hi @jeremieberrebi this is completely possible with osxphotos but will be slow as data has to go from the NAS to the Mac then back to the NAS. You'll want to use --ramdb and or --exportdb options to keep the export database in memory and store it on the Mac, respectively. See `osxphotos help export ramdb‘

Other than that everything should just work. @oPromessa is a Synology user I think so he might have some tips.

[jeremieberrebi]

Thanks a lot for your answer and for this incredible script. It's exactly what I was looking for.

Why couldn't it be possible to run the script directly on the Synology itself, since Synology is Linux-based?

[RhetTbull]

That's likely possible and would definitely be faster but I don't use a Synology so don't know where to start. Getting osxphotos onto the NAS would at a minimum require a working copy of python3 in order to run python3 -m pip install osxphotos or python3 -m pip install pipx then pipx install osxphotos. I think the Synology can run Docker containers so you could do this in a container (but would need to map the right paths so the container has access to the Photos and export location).

If you are doing this from the Mac, I also recommend the --retry option in case the NAS or the network times out. --retry 3 is a good starting value.

[RhetTbull]

By the way, if you're actively using the library from the NAS, this is something Apple recommends against as Photos expects the library database to be available on a fast locally connected drive.

[jeremieberrebi]

Oh really? I saw PowerPhotos and can buy it but I thought that it was better to export everything and reimport everything to create a real new Library from scratch.
My Library is 15 years old....

I tried to use OSXPhotos now with the
osxphotos export --skip-edited --retry 3 /Volumes/Photos\ SBackup --ramdb
command line
but I forgot to define a Directory structure. All was about to be exported in one folder...
What would you recommend?

If you tell me that it's better for me to use PowerPhotos, I'll do it.

[jeremieberrebi]

I bought PowerPhotos but unfortunately, I can't export to a Library on a NAS. (I don't have enough space to export almost 3TB of pictures and videos on my external drive nor on my Mac).
So it seems the best way will be to use OSXPhotos, export everything to the NAS, delete the Library, and reimport everything in a new Library.
Still would be happy to get your opinion
Thanks a lot again

[jeremieberrebi]

PowerPhotos is really too slow with a so big library.

Just launched a Backup with this command line
osxphotos export --skip-edited --retry 3 /Volumes/Photos\ SBackup --ramdb --export-by-date
Will see :)
Would love to read how the import options works after that.
Not sure the Photos app will not explode when I'll ask it to import so many files.

[oPromessa]

Will try to provide more input later on.

For now quickly:

###on export

... you may want to use options like --exiftool --keywords and some others to make sure all your photos edits are exported into the files. Option --sidecar xmp may also be interesting.

I'd do it on the Mac into the NAS Synology server.

###on import
Osxphotos may not be optimal (fast enough) but will import the info.

You could also import straight into Photos as it would read metadata from the files (created with exiftool on export). And maybe select to respect the folders into albums.

Some file formats don't support all metadata (some videos. PNG files which in my case I've converted them to jpg)

PS: will try to provide more details later.

[oPromessa]

Oh. Forgot. Also use the --exportdb option to have the osxphotos export control database on the local Mac disk which is much faster for the export.

[RhetTbull]

Your current export will not preserve any edited images (I assume that's intentional)? Also, you'll lose all metadata -- keywords, favorite status, albums, etc. Is this important to you? If you'd like to preserve metadata, I suggest the following:

You can read the docs to see what the options do:

Export:

osxphotos export /Volumes/Photos\ SBackup --retry 3 --ramdb --sidecar json --directory "{folder_album,NoAlbum}" --person-keyword --keyword-template "{favorite?Favorite,}"

This writes metadata to a JSON sidecar, exports to folders that match the folder/album in Photos (but puts photos in no album in "NoAlbum" folder), creates keywords for each person in the photo (persons cannot be re-imported so this at least preserves what's there...leave it off if not important to you), and creates a new keyword for all photos marked as favorites. (osxphotos cannot yet mark photos as favorites on import)

Import:

Create a new Photos library (quit Photos, then re-open while holding down Option)

osxphotos import /Volumes/Photos\ SBackup/* --walk --sidecar --skip-dups --dup-albums --album "{filepath.parent}" --relative-to /Volumes/Photos\ SBackup/ --split-folder "/"

This reads metadata from the sidecar created in step 1, skips duplicates (a photo could be in more than one album and thus a duplicate would have been exported), does add the duplicates back to the right album though (without importing a duplicate copy), creates albums and folders based on the exported path created in step 1.

[RhetTbull]

Yes. When osxphotos says "photos" it means assets which could be photo or video.

Missing could be assets in shared albums (these are often not downloaded by iCloud), referenced images (not copied to the library) that moved and can't be found, or images that were never downloaded from iCloud.

The easiest way to find them is with the export report option. Since you didn't use --report you can use the exportdb command to recreate it:

osxphotos exportdb /path/to/export --report export.csv 0

This generates the report of the export showing each file that was missing, errors, missing etc.

[jeremieberrebi]

@RhetTbull Thanks again. I'm not using iCloud for iPhotos.
When I read the export.csv file, it seems that the missing files are in different directories and for many differents periods. It's bizarre.

https://www.dropbox.com/scl/fi/bmgdzijbc5pzhxcm6ng4o/export.csv?rlkey=ify8ao65zs0ccvzvhfss2tyto&dl=0

Do I need to check them one by one to see if I can recover them?

[jeremieberrebi]

Also you didn't told me why there is a differences between "photos" and "exported" numbers in the report?

[RhetTbull]

differences between "photos" and "exported" numbers in the report?

"photos" count refers to total number of items (assets) in the Photos library. "exported" refers to the number of files osxphotos exported. Each asset could have more than one file: a live photo has photo+video, a RAW+JPEG pair has RAW and JPEG. An edited photo has original + edited.

[RhetTbull]

Do I need to check them one by one to see if I can recover them?

For the missing files, yes. I don't know why they're missing. You can find the UUID for one of the files that's missing from the repor then do something like this:

cd /path/to/export; find . | grep UUID

This will search for any files associated with the asset's UUID. For example, it might find an edited version or a preview of the file.

osxphotos has an option to export previews if missing so you could also re-run your export using --update --preview-if-missing