Image loading is slow

[SuperDisk]

Loading a 1920x1080 image with

(magick:with-magick-wand (wand)
  (time
   (magick:read-image-blob wand (alexandria:read-file-into-byte-vector "/tmp/my-img.bmp"))))

takes around 500 ms on my machine. I wrote a simple C program to do the same thing, and it completes in around 1 ms. Any idea why it would be so much slower? I did a quick glance into the code but it seems like it's a pretty thin wrapper over MagickReadImageBlob.

[SuperDisk]

Ah, I found it's because the library injects code to do a very slow byte-by-byte copy into a foreign allocated vector before passing it to ImageMagick.

(cffi:defcfun "MagickReadImageBlob" :boolean
  (wand :pointer)
  (blob :pointer)
  (length :int))


(magick:with-magick-wand (wand)
  (let ((vec (alexandria:read-file-into-byte-vector "/tmp/bigdump.bmp")))
    (cffi:with-pointer-to-vector-data (ptr vec)
      (time
       (magickreadimageblob wand ptr (length vec)))
      (magick:write-image wand "/tmp/mydump.bmp "))))

This code in contrast takes around 3ms.

[SuperDisk]

I think the function in the library, as-is also leaks memory or something, since even just a few loads can cause a blowup in heap usage. The small cffi wrapper I posted above doesn't have this problem.

[ruricolist]

Do you want to make PR with this change?

[SuperDisk]

I'd be happy to, but I'm actually not sure what the best way to do it is. I just slapped the CFFI definition in my own code and started using it, but the defmagickfun stuff is complicated and I'm not sure if it would play nice to just insert this definition. Basically any function that's declared to take a dynarray could use this optimization, it's just that the image loading seemed to be the part that was causing me really annoying slowdown.

Also, technically cffi:with-pointer-to-vector-data is specified to only take vectors made with cffi:make-shareable-byte-vector, but it happens that on SBCL cffi:make-shareable-byte-vector just calls make-array with no special behavior, so this works. On ABCL for instance it would be problematic I think.

[ruricolist]

I'm wary of making major changes to a library (1) without a test suite and (2) that I no longer personally use, but in principleb it seems like it would be a useful optimization to make defmagickfun expand to pass the array without a copy on SBCL.