Resize and Fit fail to render some of images while using '--ignoreCache'

[farynaio]

Most of the images generated by Resize and Fit are broken.

For every image in project I use partial image.html to create responsive images. Everything works fine when working on in-memory server, things breaks when generating deployment.

To make sure no stale cache interfere with the result I use hugo --minify --ignoreCache --cleanDestinationDir --gc to generate deployment. After that in public folder I get image file as expected but half of them are broken, and can’t be open in any image viewer, their size is 0 bytes - it can be verified by checking generated /public folder.

I created sample, minimal repo to reproduce this issue. All the images in posts are lowercase, kebab-case JPGs. All created in the same way using https://www.canva.com/ and later processed with GIMP. All have the same width and height.

The repo uses theme as submodule so the easiest way to clone it is:

git clone --recurse-submodules git@github.com:adamfaryna/hugo-images-test.git

Make sure to npm i

The images renders fine when hugo server --gc --disableFastRender -v.

The generated images are broken on deployment build:

hugo --minify --ignoreCache --cleanDestinationDir --gc && npm run serve

It’s Github so the default repo is “main”.

I store project in case sensitive virtual volume so even though I work on macOS the case insensitivity is not a cause of an issue.

What version of Hugo are you using (hugo version)?

Hugo Static Site Generator v0.79.0-DEV/extended darwin/amd64 BuildDate: unknown

macOS 10.15.7 Catalina

Does this issue reproduce with the latest release?

Yes

[jmooring]

I can reproduce the behavior with this:

rm -rf public/ resources/
hugo --minify --ignoreCache --cleanDestinationDir --gc 
npm run serve

But this works fine (don't ignore cache):

rm -rf public/ resources/
hugo --minify --cleanDestinationDir --gc 
npm run serve

[farynaio]

I can reproduce the problem with this:

rm -rf public/ resources/
hugo --minify --ignoreCache --cleanDestinationDir --gc 
npm run serve

But this works fine (don't ignore cache):

rm -rf public/ resources/
hugo --minify --cleanDestinationDir --gc 
npm run serve

@jmooring thanks!

From my perspective it looks like Fit & Resize generate broken images when working without cache together with with above of certain number of images. When I was using only 4 images everything worked fine, even though cache was turned off.

[bep]

hugo --minify --ignoreCache --cleanDestinationDir --gc

While we need to fix this particular issue, running with --ignoreCache is rather coarse grained, esp. considering images. There is a "cache config" where you can "ignore the caches" you don't like (for some reason).

[bep]

Also, a note to whoever fixes these:

• The likely cause of this is file read/write concurrency. I have tried to avoid introducing file locking, but that may be needed here.
• I have a "under progress" pull request where you can limit the memory cache (which I also plan to use for .Content) which should greatly improve the memory usage, esp. in low memory situations.

[EmpireJones]

I was running into this issue today. At first I thought it was #7955, as both result in zero-length output.

"ignoreCache" and disabling cache via maxAge sound like safe things to do, from the user perspective, so it may be worth adding a note in the docs, or a warning of some sort, until this is fixed.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. The resources of the Hugo team are limited, and so we are asking for your help.
If this is a bug and you can still reproduce this error on the master branch, please reply with all of the information you have about it in order to keep the issue open.
If this is a feature request, and you feel that it is still relevant and valuable, please tell us why.
This issue will automatically be closed in the near future if no further activity occurs. Thank you for all your contributions.

[bep]

@Setr43 please write in English here.-

[olegsinavski]

I’m getting the same issue - images are being processed to 0 bytes size. Somehow, it depends on the exact size and the format (jpg/png). Removing the --ignoreCache flag helped me as well.
Version: hugo v0.105.0+extended darwin/amd64 BuildDate=unknown.
Code:

{{ $img := resources.Get "images/my_image.png" }}
{{ $img := $img.Fit "450x460" }}

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. The resources of the Hugo team are limited, and so we are asking for your help.
If this is a bug and you can still reproduce this error on the master branch, please reply with all of the information you have about it in order to keep the issue open.
If this is a feature request, and you feel that it is still relevant and valuable, please tell us why.
This issue will automatically be closed in the near future if no further activity occurs. Thank you for all your contributions.

[toby3d]

Bumb, as the problems are reproduced especially when working with images in Git LFS.

[mrled]

I ran into this issue today. In my case, it was triggered by having two or more pages with no title set in their front matter, and using --ignoreCache.

I created a small proof of concept site that shows the behavior: https://github.com/mrled/hugo-test-ogimage. It's as simple as I could make it, with the minimum number of pages and layouts needed.

I was interested to find that it's not consistent, such that sometimes the site generates just fine several times in a row before failing, so I started running my test builds in a loop.

When I realized I had a problem, it was hard to track down the cause. I ran into this trying to add automatically generated og:image files to my site, so every page on my site was generating an image, and all I knew was that one page sometimes generated a zero-byte file instead. I had to hack my site apart bit by bit before I found the few pages with no title in their front matter that caused the problem. I had also included --ignoreCache in my Makefile at some point in the past, far distant from my current problem which appeared to have to do with my new use of image filters. It took building a new site from hugo init site . before I realized that --ignoreCache was the culprit.

The following probably won't benefit other users, who now know just not to pass --ignoreCache, but I did write a checkimg.sh script that searches for zero-byte files in the generated image path. Adapt to your site and run after running hugo, maybe in your Makefile or similar, and it'll exit with a failure if it finds any zero-byte files.

+1 the request to fix this or remove --ignoreCache altogether. If those aren't possible in the short term, might you add a warning when passing --ignoreCache instead? Such a warning would have saved me time as I would have seen it in the build logs as soon as I noticed any problems.

[jmooring]

Verified that this is fixed by #11894.

[jmooring]

Fixed with #11894

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.