improved handling of animated images #357

BPplays · 2024-08-21T22:39:23Z

I've changed the handling of animated images to include support for more types of animated images.
PR features:

includes transcoding from an animated image type to webp or gif using pillow.
only counts an image as animated if it has more then 1 frame
checks what image types are support by qt and by pillow, as well as includes a list of known good types what pillow can encode with animation, and sorts available image types by quality

this PR also includes the changes from #344 so feel free to close that one if you prefer this one, though sadly that plugin doesn't seem to support decoding animated jxls

Closes #333

tagstudio/src/qt/widgets/preview_panel.py

BPplays · 2024-08-22T13:54:13Z

i added some code to test converting into animated webp before showing the image but it's sometimes causing crashing and i have no idea why, when it works for a small amount of time it seems to work.

BPplays · 2024-08-22T17:53:20Z

~~i think i might have fixed the crashing~~
it hasn't crashed once after testing a lot while developing it more, so im gonna call it fixed

CyanVoxel · 2024-09-01T20:33:39Z

Wanted to chime in here with the state of this PR and other relevant developments around the project.

#344 was at a sticking point regarding licenses, but is probably going to move forward with pillow-jpegxl-plugin assuming it functions correctly (I need to get my hands on some .jxl files).

In the meantime, I've unknowingly implemented some of changes here in #409 - mostly the animated image media type category and some code for loading GIFs into memory rather than streaming them from disk.

I would also suggest rebasing this to Alpha-v9.4, however there's likely due to be more than a few merge conflicts... Some of this is due to the changes mentioned above, and others are due to some git mishaps that happened in a recent update of Alpha-v9.4. But with the rest of the Alpha v9.4 milestone features out of the way, this should have a clear path on that branch now (outside of #344 presumably getting merged).

BPplays · 2024-09-01T22:04:00Z

@CyanVoxel i manually merged the work from the older version of this branch on to Alpha-v9.4
edit: here is a jxl

eivl · 2024-09-06T18:39:31Z

Are no tests affected by this PR after the gif changes? And is there a way to add missing tests?
I don't see anything that makes me think this would not work; I would also like a short description of how to test this live on my machine.

BPplays · 2024-09-08T03:34:59Z

Are no tests affected by this PR after the gif changes?

@eivl are there any tests for animated image handling?
i couldn't find any after a quick look in ./tagstudio/tests in the Alpha-v9.4 branch.

And is there a way to add missing tests?

probably

Co-Authored-By: BPplays <58504799+bpplays@users.noreply.github.com>

BPplays · 2024-10-21T17:40:30Z

@CyanVoxel i think the PR is good enough for you to have another look if you want.
i also moved setting what preview media type to display to it's own function.

CyanVoxel · 2024-10-25T19:31:30Z

tagstudio/src/qt/widgets/preview_panel.py

+                            if hasattr(image, "n_frames"):
+                                if image.n_frames > 1:
+                                    self.set_anim_img(filepath, ext, file_bytes, image)


Just an approval comment: I like the check to see if an image has multiple frames - for some reason Qt displays QMovies a bit strangely compared to static images, so this check is nice for improving the quality of non-animated images that share extensions with animated counterparts (i.e. gif).

CyanVoxel · 2024-10-25T19:44:03Z

tagstudio/src/qt/widgets/preview_panel.py

+        ani_img_priority_order = ["jxl", "apng", "png", "webp", "gif"]
+
+        self.preview_anim_img_pil_map = {"apng": "png"}
+
+        self.preview_anim_img_pil_map_args = {"gif": {"disposal": 2}}
+
+        self.preview_anim_img_pil_known_good = {"webp", "gif"}


There was a lot of this code that I opted to leave out when porting the functionality over due to it being difficult to decipher the purpose of and to trace back what was occurring inside the animated image creation block.

If it's necessary for important additional functionality, I would appreciate some documentation on what exactly all of these maps are being used for and why they need to be used over just loading in the image as a gif/webp.

@CyanVoxel this is for choosing image format priority based on what types are supported for any given version of pillow and qt

preview_anim_img_fmts: this is a list of qmovie supported formats, it also gets sorted base on:

ani_img_priority_order: is an order of image extensions that are sorted first in preview_anim_img_fmts

preview_anim_img_pil_map: this maps what extension to use in pillow, pillow encodes animated pngs with the format 'png' instead of 'apng' so if apng comes up anywhere pillow should use png

preview_anim_img_pil_map_args: this is special args pillow should use for specific encoding formats, here gif has a trail on transparency if 'disposal' is not 2, but webp doesn't need it.

preview_anim_img_pil_known_good: it's a list of image types pillow can encode with animation, ~~i couldn't find a way to test this in the program, it also probably needs expanding~~ i made a commit, preview_anim_img_pil_known_good is also now preview_anim_img_pil_anim_supported

the logic to choose image formats happens in get_anim_ext

…ave less of an implication of a permanent conversion

BPplays · 2024-10-26T14:08:59Z

@CyanVoxel it seems like it blocks the gui thread while it's saving an image causing quite a bit of lag for large-ish images,
this should be fixed by saving the image, waiting for it to be done then loading it without blocking the gui but im not sure how to do that.
i've made a basic and kinda janky threaded implementation with in memory caching but, it still has some issues for example: it doesn't update to the animated image until you go back for transcoded images.
also multiprocessing might work better for this.

…nd partially port #357) (#549) * feat: add JXL image thumbnail support Co-Authored-By: BPplays <58504799+bpplays@users.noreply.github.com> * feat: add animated previews for webp and apng Co-Authored-By: BPplays <58504799+bpplays@users.noreply.github.com> --------- Co-authored-by: BPplays <58504799+bpplays@users.noreply.github.com>

BPplays force-pushed the jxl_thumbnails branch from 2c4e35d to 2a15886 Compare August 21, 2024 22:50

BPplays mentioned this pull request Aug 22, 2024

does this support animated jxl? inflation/jpegxl-rs#78

Open

Thesacraft reviewed Aug 22, 2024

View reviewed changes

tagstudio/src/qt/widgets/preview_panel.py Outdated Show resolved Hide resolved

BPplays marked this pull request as ready for review August 22, 2024 21:38

CyanVoxel added Type: Enhancement New feature or request Type: UI/UX User interface and/or user experience Status: Review Needed A review of this is needed labels Aug 22, 2024

CyanVoxel added this to the Alpha 9.5 milestone Aug 23, 2024

BPplays mentioned this pull request Aug 24, 2024

[Feature Request]: Make gifs animate in app #338

Closed

3 tasks

CyanVoxel added the Priority: Medium An issue that shouldn't be be saved for last label Sep 1, 2024

BPplays force-pushed the jxl_thumbnails branch from ce2dc6c to 3fe7d4d Compare September 1, 2024 22:01

BPplays changed the base branch from thumbnails to Alpha-v9.4 September 1, 2024 22:02

CyanVoxel removed the Status: Review Needed A review of this is needed label Sep 15, 2024

CyanVoxel changed the base branch from Alpha-v9.4 to main October 11, 2024 22:33

CyanVoxel and others added 2 commits October 17, 2024 15:49

feat: add JXL image thumbnail support

28a970c

Co-Authored-By: BPplays <58504799+bpplays@users.noreply.github.com>

feat: add animated previews for webp and apng

878a873

Co-Authored-By: BPplays <58504799+bpplays@users.noreply.github.com>

CyanVoxel mentioned this pull request Oct 19, 2024

feat: add JXL thumbnail and animated APNG + WEBP support (port #344 and partially port #357) #549

Merged

BPplays force-pushed the jxl_thumbnails branch from 3fe7d4d to 878a873 Compare October 20, 2024 07:17

BPplays added 6 commits October 20, 2024 00:48

added more checks and file ext options

1bfd2e7

fixed frame number check

2572f4d

fixed args

c3dd683

ruff format

0a23e21

optimized file reads

fcc0606

changed variable names

c622a52

BPplays added 3 commits October 21, 2024 05:52

formatting changes

98b6f09

ruff format

83fd20a

changed code to have less duplication

6b00f28

CyanVoxel reviewed Oct 25, 2024

View reviewed changes

BPplays added 11 commits October 25, 2024 20:51

added to preview_anim_img_pil_known_good

a4a08db

added to preview_anim_img_pil_known_good

74ec8e1

programmatically checked what image types pillow supports

e096371

changed stuff for mypy and ruff

da28dbc

changed wording on logs from 'converting to' to 'transcoding to' to h…

38b4544

…ave less of an implication of a permanent conversion

changed sort order

51fc921

changed sort order

c61b617

added comments

e942d55

ruff format

a955676

added comments

c3ebae1

fixed gif being the wrong size for one-ish frame

acae7da

BPplays added 9 commits October 26, 2024 08:16

testing threading

f1ac673

done base threading

e168655

made code better

9f1e87b

fixed bug

fc225a1

testing mp

ba7a153

testing

314157a

an actually ok threading implimentation

6a60fad

an actually ok threading implimentation

f880dc0

fixed caching issues

0129e07

CyanVoxel added the Status: Changes Requested Changes are quested to this label Dec 10, 2024

CyanVoxel removed this from the Alpha v9.5 (Post-SQL) milestone Dec 21, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

improved handling of animated images #357

improved handling of animated images #357

BPplays commented Aug 21, 2024 •

edited

Loading

BPplays commented Aug 22, 2024

BPplays commented Aug 22, 2024 •

edited

Loading

CyanVoxel commented Sep 1, 2024

BPplays commented Sep 1, 2024 •

edited

Loading

eivl commented Sep 6, 2024

BPplays commented Sep 8, 2024 •

edited

Loading

BPplays commented Oct 21, 2024

CyanVoxel Oct 25, 2024

CyanVoxel Oct 25, 2024

BPplays Oct 26, 2024 •

edited

Loading

BPplays commented Oct 26, 2024 •

edited

Loading

improved handling of animated images #357

Are you sure you want to change the base?

improved handling of animated images #357

Conversation

BPplays commented Aug 21, 2024 • edited Loading

BPplays commented Aug 22, 2024

BPplays commented Aug 22, 2024 • edited Loading

CyanVoxel commented Sep 1, 2024

BPplays commented Sep 1, 2024 • edited Loading

eivl commented Sep 6, 2024

BPplays commented Sep 8, 2024 • edited Loading

BPplays commented Oct 21, 2024

CyanVoxel Oct 25, 2024

Choose a reason for hiding this comment

CyanVoxel Oct 25, 2024

Choose a reason for hiding this comment

BPplays Oct 26, 2024 • edited Loading

Choose a reason for hiding this comment

BPplays commented Oct 26, 2024 • edited Loading

BPplays commented Aug 21, 2024 •

edited

Loading

BPplays commented Aug 22, 2024 •

edited

Loading

BPplays commented Sep 1, 2024 •

edited

Loading

BPplays commented Sep 8, 2024 •

edited

Loading

BPplays Oct 26, 2024 •

edited

Loading

BPplays commented Oct 26, 2024 •

edited

Loading