Pillow raises IOError on thumbnail generation

[bryab]

Description

When attempting to upload an image to a room through the Riot Android app, a server error occurs which prevents the image from being uploaded. It appears Pillow is raising an IOError.

Steps to reproduce

• On Android, attempt to "share" an image to Riot, and select a room.

I receive an infinitely repeating "Internal server error", and I must force-stop the Riot app to recover from it. Below is the server error.

synapse.http.server - 102 - ERROR - POST-36609 - Failed handle request via <function _async_render_POST at 0x7fd81b836938>: <SynapseRequest at 0x7fd8183d3320 method=POST uri=/_matrix/media/v1/upload?access_token=<redacted>&filename=15279022308169654444447737565.jpg clientproto=HTTP/1.0 site=8008>: Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 653, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1442, in gotResult
    _inlineCallbacks(r, g, deferred)
  File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1384, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/local/lib/python2.7/dist-packages/twisted/python/failure.py", line 422, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
--- <exception caught here> ---
  File "/usr/local/lib/python2.7/dist-packages/synapse/http/server.py", line 79, in wrapped_request_handler
    yield h(self, request)
  File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1384, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/local/lib/python2.7/dist-packages/twisted/python/failure.py", line 422, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/local/lib/python2.7/dist-packages/synapse/rest/media/v1/upload_resource.py", line 97, in _async_render_POST
    content_length, requester.user
  File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1384, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/local/lib/python2.7/dist-packages/twisted/python/failure.py", line 422, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/local/lib/python2.7/dist-packages/synapse/rest/media/v1/media_repository.py", line 164, in create_content
    None, media_id, media_id, media_type,
  File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1384, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/local/lib/python2.7/dist-packages/twisted/python/failure.py", line 422, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/local/lib/python2.7/dist-packages/synapse/rest/media/v1/media_repository.py", line 616, in _generate_thumbnails
    t_width, t_height, t_type,
  File "/usr/local/lib/python2.7/dist-packages/twisted/python/threadpool.py", line 250, in inContext
    result = inContext.theWork()
  File "/usr/local/lib/python2.7/dist-packages/twisted/python/threadpool.py", line 266, in <lambda>
    inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
  File "/usr/local/lib/python2.7/dist-packages/twisted/python/context.py", line 122, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/usr/local/lib/python2.7/dist-packages/twisted/python/context.py", line 85, in callWithContext
    return func(*args,**kw)
  File "/usr/local/lib/python2.7/dist-packages/synapse/rest/media/v1/thumbnailer.py", line 60, in scale
    return self._encode_image(scaled, output_type)
  File "/usr/local/lib/python2.7/dist-packages/synapse/rest/media/v1/thumbnailer.py", line 96, in _encode_image
    output_image.save(output_bytes_io, self.FORMATS[output_type], quality=80)
  File "/usr/local/lib/python2.7/dist-packages/PIL/Image.py", line 1935, in save
    save_handler(self, fp, filename)
  File "/usr/local/lib/python2.7/dist-packages/PIL/JpegImagePlugin.py", line 622, in _save
    raise IOError("cannot write mode %s as JPEG" % im.mode)
exceptions.IOError: cannot write mode RGBA as JPEG

Version information

• Homeserver: Mine
• Version: Synapse/0.30.0
• Install method: docker image: https://github.com/silvio/docker-matrix
• Platform: Debian 9 host running docker. Matrix served thru nginx proxy.

[andrewshadura]

I had a similar, but different issue which I fixed by bumping Pillow to 4.3.0.

  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1355, in gotResult
    _inlineCallbacks(r, g, deferred)
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1299, in _inlineCallbacks
    result = g.send(result)
  File "/usr/lib/python2.7/dist-packages/synapse/rest/media/v1/media_repository.py", line 165, in create_content

    None, media_id, media_id, media_type,
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1445, in unwindGenerator
    return _inlineCallbacks(None, gen, Deferred())
--- <exception caught here> ---
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1299, in _inlineCallbacks
    result = g.send(result)                                                                                    
  File "/usr/lib/python2.7/dist-packages/synapse/rest/media/v1/media_repository.py", line 583, in _generate_thumbnails

    thumbnailer = Thumbnailer(input_path)
  File "/usr/lib/python2.7/dist-packages/synapse/rest/media/v1/thumbnailer.py", line 32, in __init__
    self.image = Image.open(input_path)
  File "/usr/lib/python2.7/dist-packages/PIL/Image.py", line 2257, in open
                                                                                                               
  File "/usr/lib/python2.7/dist-packages/PIL/Image.py", line 350, in preinit
    def preinit():
  File "/usr/lib/python2.7/dist-packages/PIL/GifImagePlugin.py", line 775, in <module>
    Image.register_save_all(GifImageFile.format, _save_all)
exceptions.AttributeError: 'module' object has no attribute 'register_save_all'                                

[mmueller]

@andrewshadura's issue is not the same issue reported in this ticket.

The RGBA issue is not fixed by upgrading Pillow. I run into the same server with Pillow 5.2.0 on my homeserver 0.33.0. It happens only when uploading PNGs and I believe the reason is that the thumbnailer needs to flatten the image from the RGBA space to RGB before attempting to write a JPG thumbnail image. Either that or generate PNG thumbnails.

Pillow previously allowed you to do this with a warning, but now it's an error.

See: https://github.com/python-pillow/Pillow/blob/master/docs/releasenotes/4.2.0.rst#removed-deprecated-items

[richvdh]

fixed by #6241