Add HEIF/HEIC/AVIF upload support

[Ruin0x11]

Adds support for uploading HEIF/HEIC/AVIF files using libheif and libavif. Chromium appears to request this format by default on some sites (hitomi.la) and in Firefox support can be enabled with a config option.

Because FFmpeg does not support HEIF, any HEIF format images are converted to PNG before being passed to FFmpeg if extra processing is needed (like for thumbnail generation). But the original file is preserved as the post's content.

Also, it seems that support for displaying certain HEIF images inside img tags is still limited even inside Chromium, but the primary use case is for downloading HEIF images that are being properly displayed on other sites and uploading them to szuru.

Closes #398.

[sgsunder]

It's a good idea, but I'm thinking we should maybe pause on supporting it until it has better support in ffmpeg and browsers?

I'm conflicted on this, any thoughts from @rr- or @neobooru ?

[rr-]

To be honest I don't see any harm in merging this, especially considering how it's well tested and doesn't introduce any changes or regressions – only a new feature.

[Ruin0x11]

I did some research and it turns out that AVIF is the only format out of the three that's starting to be supported in web browsers.

HEIC/HEIF cannot be embedded with img, only canvas and some specialized rendering. However HEIC is being used as the default file format for the camera app in recent Apple devices (it's converted to JPG on the fly for file picker APIs and similar).

So as it stands HEIC/HEIF files won't be displayed properly (yet), only the thumbnails.

However it could be the case that the sheer number of iPhone users will cause the format to gain ubiquity. See python-pillow/Pillow#2806 (comment).

FFmpeg support is unlikely to happen overnight as it's been an open issue for 4 years. The format itself is complicated to implement in its entirety since it includes things like composite images (HDR) wrapped up in a large number of extensions to ISO standards.

[neobooru]

I personally have already encountered a few images which where in AVIF format, so I already had that on my backlog. It looks like most major browsers (except safari, of course) support it. Starting on the first of june Firefox will have AVIF support enabled by default (atm it is still opt-in).

As for the HEIC stuff (which I assume is standalone from AVIF?) I don't really see an use case for it atm, because the image display doesn't work, but I guess it also doesn't hurt. I can install all required dependencies on my Debian stable, which means that most users who deploy manually should still be able to do so.

I also read the FFmpeg thread, and it doesn't look like that is going to change anytime soon.

[neobooru]

Seeing as there are no objections against this I'll (try to) merge it this weekend. Same goes for your other PRs.

[neobooru]

I had to make a few changes to get this to work. You can see those in #413, please post any questions/objections you might have.

[Giraffaman]

Hi - sorry I've just started trying out szurubooru but I'm getting internal server errors when trying to upload any .mp4 file, with the following being logged in the server-container's log:

[2021-06-06 00:55:35] waitress Exception while serving /posts/reverse-search
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/waitress/channel.py", line 350, in service
    task.service()
  File "/usr/lib/python3.8/site-packages/waitress/task.py", line 171, in service
    self.execute()
  File "/usr/lib/python3.8/site-packages/waitress/task.py", line 441, in execute
    app_iter = self.channel.server.application(environ, start_response)
  File "/opt/app/szurubooru/rest/app.py", line 104, in application
    response = handler(ctx, match.groupdict())
  File "/opt/app/szurubooru/api/post_api.py", line 295, in get_posts_by_image
    lookalikes = posts.search_by_image(content)
  File "/opt/app/szurubooru/func/posts.py", line 931, in search_by_image
    query_signature = image_hash.generate_signature(image_content)
  File "/opt/app/szurubooru/func/image_hash.py", line 231, in generate_signature
    im_array = _preprocess_image(content)
  File "/opt/app/szurubooru/func/image_hash.py", line 45, in _preprocess_image
    img = Image.open(BytesIO(content))
  File "/usr/lib/python3.8/site-packages/PIL/Image.py", line 2916, in open
    im = _open_core(fp, filename, prefix)
  File "/usr/lib/python3.8/site-packages/PIL/Image.py", line 2902, in _open_core
    im = factory(fp, filename)
  File "/usr/lib/python3.8/site-packages/PIL/ImageFile.py", line 117, in __init__
    self._open()
  File "/usr/lib/python3.8/site-packages/pyheif_pillow_opener/__init__.py", line 19, in _open
    heif_file = pyheif.read(self.fp)
  File "/usr/lib/python3.8/site-packages/pyheif/reader.py", line 37, in read
    result = _read_heif_bytes(d, apply_transformations, convert_hdr_to_8bit)
  File "/usr/lib/python3.8/site-packages/pyheif/reader.py", line 66, in _read_heif_bytes
    raise ValueError("Input is not a HEIF/AVIF file")
ValueError: Input is not a HEIF/AVIF file

I've tried mp4-files which were downloaded with youtube-dl, as well as files which were downloaded through Firefox.

Is this related to the discussion in here? If so, is there any way one can make sure that some mp4 file can be uploaded, like converting it in some way?

[neobooru]

@Giraffaman It looks like this PR broke the mp4 upload in some way. I'll take a look at it asap!

[neobooru]

@Giraffaman Could you try again? The newest version (also published to docker hub) should have fixed this issue.

[Giraffaman]

@neobooru looks good - I removed the client, server and sql container and recreated them, pulling the latest image. Now I am able to upload any mp4 file and also playback works like a charm.