Ruby script crashes on Windows when using hist_find

[niklasweber]

Hi, I am having an issue with ruby-vips on windows. When I run some_image.hist_find it seems like ruby crashes. I condensed my problem down to the following script which reliably crashes each time for me:

require 'vips'

img = Vips::Image.new_from_file('test.jpg', access: :sequential)
puts('Loaded image')
img.hist_find
puts('Ran hist_find')

start_time = Time.now
while true
    puts "Elapsed: #{ Time.now - start_time} seconds"
end

The output is:

Loaded image
Ran hist_find
Elapsed: 0.0 seconds
[omitted]
Elapsed: 0.5107214 seconds
Elapsed: 0.5107635 seconds
Elapsed: 0.5108055 seconds
Elapsed: 0.5108474 seconds
[crash]

The expected behavior would be that the for loop runs forever, but after around 0.5 seconds it just stops without any other output and $? returns false. I initially used a tiff image but the problem persists also if I use jpeg or other image formats. Also very small and very large images do not make a difference. Removing access: :sequential from new_from_file in order to use the default VIPS_ACCESS_RANDOM method also still makes it crash. The problem only occurs if I run hist_find. When leaving out img.hist_find it runs fine.

This problem only occurs on Windows. On Linux it runs fine.

My setup:
Windows 10
ruby-vips 2.1.4
ruby 2.7.5p203 (2021-11-24 revision f69aeb8314) [x64-mingw32]

To install ruby-vips I put

gem 'ruby-vips', '~> 2.0', '>= 2.0.7', :require => false

in my Gemfile and ran

bundler install

Can someone reproduce this? What could be the problem here?
Thank you very much!

[jcupitt]

Hi @niklasweber,

Sorry you're having trouble. How did you install ruby, and do you know which msys you have? I know I've had difficulty with the various windows ruby installers in the past.

[jcupitt]

Oh, and I guess this is with the libvips binary that gem pulls in for you, is that right?

[jcupitt]

I tried with "Ruby+Devkit 3.1.1-1 (x64)" from:

https://rubyinstaller.org/downloads/

I ran gem install ruby-vips (ie. using the libvips binary in pacman) and I see an exit after 0.5s as well. I'll try with some other libvips binaries.

[jcupitt]

Using the official libvips windows binaries seems to fix this. I made a tiny test prog:

require 'vips'

puts "ruby-vips version = #{Vips::VERSION}"
puts "libvips version = #{Vips::LIBRARY_VERSION}"

Just after gem install ruby-vips I see:

ruby-vips version = 2.1.4
libvips version = 8.12.1-Wed Nov 24 15:41:04 UTC 2021

I downloaded the official libvips zip:

https://github.com/libvips/build-win64-mxe/releases/download/v8.12.2-build2/vips-dev-w64-all-8.12.2.zip

And unzipped it over the top of /c/Ruby31-x64/msys64/ucrt64. I now see:

ruby-vips version = 2.1.4
libvips version = 8.12.2-Tue Jan 25 09:34:32 UTC 2022

And your program works fine.

[jcupitt]

(this is not a great solution -- it would be better to fix the libvips package in pacman)

[niklasweber]

Hi @jcupitt,

Thanks for checking up on my issue. Regarding your questions: I installed Ruby+Devkit 2.7.5-1 (x64) with all default options selected. For the ridk tool I chose options 1 and 3 (ridk including the MSYS2 and the MinGW toolchains). Due to other dependencies I cannot update to Ruby 3. I use msys2. The output of your script was

ruby-vips version = 2.1.4
libvips version = 8.11.2-Sat Jul  3 14:17:10 UTC 2021

then I ran pacman -Sy, pacman -Rns mingw-w64-x86_64-libvips and pacman -S mingw-w64-x86_64-libvips in the msys2 terminal and now the output is

ruby-vips version = 2.1.4
libvips version = 8.12.1-Wed Nov 24 15:41:04 UTC 2021

but it still crashes, like you already tested.

However trying your workaround somehow doesn't work for me.
I extracted version 8.12.2-build2 like you mentioned above to my /c/Ruby31-x64/msys64/ucrt64 but it my version is still the same:

ruby-vips version = 2.1.4
libvips version = 8.12.1-Wed Nov 24 15:41:04 UTC 2021

Did I maybe forget something?

[jcupitt]

You need to find out where your current libvips-42.dll is kept and replace that (and also replace all the associated DLLs). It's probably not in C:\ruby31.

[niklasweber]

Sorry, copy, paste error. Of course in my case I copied it to /C/Ruby27-x64/msys64/ucrt64

[jcupitt]

Maybe you have two copies of the libvips-42.dll then? It seems it's still finding the old version.

[niklasweber]

Okay it's working now. I used pacman -Ql mingw-w64-x86_64-libvips from within msys2.exe and it showed me where the files are installed.

$ pacman -Ql mingw-w64-x86_64-libvips
mingw-w64-x86_64-libvips /mingw64/
mingw-w64-x86_64-libvips /mingw64/bin/
mingw-w64-x86_64-libvips /mingw64/bin/batch_crop
mingw-w64-x86_64-libvips /mingw64/bin/batch_image_convert
mingw-w64-x86_64-libvips /mingw64/bin/batch_rubber_sheet
mingw-w64-x86_64-libvips /mingw64/bin/libvips-42.dll
mingw-w64-x86_64-libvips /mingw64/bin/libvips-cpp-42.dll
mingw-w64-x86_64-libvips /mingw64/bin/light_correct
...

$ cd /mingw64/
$ pwd -W
C:/Ruby27-x64/msys64/mingw64

So I created a backup of C:/Ruby27-x64/msys64/mingw64 and copied the v8.12.2-build2 files into there and I get the new version and the code is working fine like you stated before.

ruby-vips version = 2.1.4
libvips version = 8.12.2-Tue Jan 25 09:34:32 UTC 2022

At least the workaround is working now and I hope the pacman package can be updated soon.
Thanks for your help!

[jcupitt]

Great! I wonder what the issue is? There were no bugs fixed around histograms, so I suppose it must be a general build issue in that package. Anyway, hopefully it'll be resolved soon.

[tknerr]

Hi @jcupitt , I'm trying to reproduce the fix, but instead of extracting vips-dev-w64-all-8.12.2.zip over the existing C:/Ruby27-x64/msys64/mingw64 directory, I wanted to place it in a separate directory and point to that via RUBY_DLL_PATH.

I'm using rubyinstaller-devkit-2.7.5-1-x64.exe, which should support RUBY_DLL_PATH, but seemingly I can't get it to work :-/

I have in C:\Users\tknerr\test.rb the version debug output from above, additionally outputting the RUBY_DLL_PATH env var:

require 'vips'

puts "RUBY_DLL_PATH is #{ENV['RUBY_DLL_PATH'].inspect}"
puts "ruby-vips version = #{Vips::VERSION}"
puts "libvips version = #{Vips::LIBRARY_VERSION}"

The vips-dev-w64-all-8.12.2.zip has been extracted to /c/Users/tknerr/Downloads/vips-dev-8.12

In cmd.exe, I'm setting RUBY_DLL_PATH now, but it still seems to load the old libvips 8.11.2 version, not the libvips 8.12.2 version:

C:\Users\tknerr>set RUBY_DLL_PATH=C:/Users/tknerr/Downloads/vips-dev-8.12/bin

C:\Users\tknerr>echo %RUBY_DLL_PATH%
C:/Users/tknerr/Downloads/vips-dev-8.12/bin

C:\Users\tknerr>ruby test.rb
RUBY_DLL_PATH is "C:/Users/tknerr/Downloads/vips-dev-8.12/bin"
ruby-vips version = 2.1.4
libvips version = 8.11.2-Sat Jul  3 14:17:10 UTC 2021

Any ideas what I might be doing wrong here?

[jcupitt]

Hi @tknerr, I tried RUBY_DLL_PATH too and couldn't get it to work.

Perhaps rubyinstaller is blocking it? You'd need to dig into the sources to see what's happening.

[tknerr]

@jcupitt thanks for double-checking! So yes, I did dig deeper... :D

It seems that an existing libvips-42.dll in C:/Ruby27-x64/msys64/mingw64/bin will always be preferred over the location specified by %RUBY_DLL_PATH%!

Check: libvips was still installed:

C:\Users\tknerr>ridk exec pacman -Qk mingw-w64-x86_64-libvips
mingw-w64-x86_64-libvips: 107 total files, 0 missing files

So I removed it:

C:\Users\tknerr>ridk exec pacman -Rns mingw-w64-x86_64-libvips
...

Confirming it's no longer installed:

C:\Users\tknerr>ridk exec pacman -Qk mingw-w64-x86_64-libvips
error: package 'mingw-w64-x86_64-libvips' was not found

Et voila, now it's picking up the correct libvips 8.12.2 from the RUBY_DLL_PATH:

C:\Users\tknerr>set RUBY_DLL_PATH=C:/Users/tknerr/Downloads/vips-dev-8.12/bin

C:\Users\tknerr>ruby test.rb
RUBY_DLL_PATH is "C:/Users/tknerr/Downloads/vips-dev-8.12/bin"
ruby-vips version = 2.1.4
libvips version = 8.12.2-Tue Jan 25 09:34:32 UTC 2022

I'm just afraid that the "msys2_mingw_dependencies" => "libvips" in the ruby-vips.gemspec will re-install it again when updating or newly installing ruby-vips... let me double-check on that next...

[jcupitt]

Ah good find. Perhaps they have some code that does something like RUBY_DLL_PATH=$MINGWHOME/bin:$RUBY_DLL_PATH. Yes, I think it's likely to break next time you update :( not ideal.

The best fix would be to get the pacman package updated, of course.

[tknerr]

Just to confirm: yes, with every ruby-vips gem installation or upgrade the libvips msys2_mingw_dependency would be reinstalled again if missing.

As we are using bundler to install ruby-vips along with other gems in a consistent / reproducible version (via frozen Gemfile.lock), and there is this commit, I see no chance to bypass installing msys2_mingw_dependencies when using bundler...

So the only option that remains would be a manual ridk exec pacman -Rns mingw-w64-x86_64-libvips after every bundle install, if we want to use a very specific / pinned libvips version via RUBY_DLL_PATH.

Having the pacman package updated would fix the original issue of this PR, but it would still not allow to use a reproducible / specific / pinned external libvips version... (but that's another topic -- should we open a separate issue to track the conversation about that?)

[jcupitt]

OK, let's close this and switch to the linked enhancement issue.