fs.outputFileSync issue with sharp 33.1 - worked in 32.6

[rdyar]

Possible bug

Is this a possible bug in a feature of sharp, unrelated to installation? YES

• [x ] Running npm install sharp completes without error.
• [ x] Running node -e "require('sharp')" completes without error.

Are you using the latest version of sharp? YES

• [x ] I am using the latest version of sharp as reported by npm view sharp dist-tags.latest.

What is the output of running npx envinfo --binaries --system --npmPackages=sharp --npmGlobalPackages=sharp?

System:
OS: Windows 11 10.0.22621
CPU: (16) x64 AMD Ryzen 7 3700X 8-Core Processor
Memory: 16.97 GB / 31.95 GB
Binaries:
Node: 20.10.0 - C:\Program Files\nodejs\node.EXE
npm: 10.2.3 - C:\Program Files\nodejs\npm.CMD
npmPackages:
sharp: ^0.33.1 => 0.33.1

What are the steps to reproduce?

I am looping over files in a folder and using fs.outpurFileSync() to resave the files - awaiting a buffer. With the prior version of sharp this works fine (32.6) but on 0.33.1 it results in an error.

Error: UNKNOWN: unknown error, open 'D:\Repos\sharp-test\images\originals\teal3.jpg'
    at Object.openSync (node:fs:581:18)
    at Object.writeFileSync (node:fs:2342:35)
    at Object.outputFileSync (D:\Repos\sharp-test\node_modules\fs-extra\lib\output-file\index.js:25:6)
    at stripRotationData (D:\Repos\sharp-test\index.js:25:8) {
  errno: -4094,
  code: 'UNKNOWN',
  syscall: 'open',
  path: 'D:\\Repos\\sharp-test\\images\\originals\\teal3.jpg'

What is the expected behaviour?

no error

Please provide a minimal, standalone code sample, without other dependencies, that demonstrates this problem

repo here: https://github.com/rdyar/sharp-test

but this is the code, fairly short:

const fs = require("fs-extra");
const path = require("path");
const sharp = require("sharp");

async function stripRotation(filePath) {
  console.log(`Processing inside strip ${filePath}`);
  const mainImage = sharp(filePath, {
    limitInputPixels: false,
    failOnError: false,
  });
  mainImage.jpeg({ quality: 98, chromaSubsampling: "4:4:4" });
  return mainImage.toBuffer();
}

const stripRotationData = async (outputFolder) => {
  const destinationFolder = path.join(outputFolder, "originals");

  const files = fs.readdirSync(destinationFolder);

  for (const file of files) {
    console.log(`Processing ${file}`);
    const filePath = path.join(destinationFolder, file);
    // eslint-disable-next-line no-await-in-loop
    const buffer = await stripRotation(filePath);
    fs.outputFileSync(filePath, buffer);
  }
};

stripRotationData("./images");

Please provide sample image(s) that help explain this problem

samples in repo, doesn't matter what the images are.

If I wrap that last line - fs.outputFileSync in a settimeout for 100 or more ms it seems to work. There seems to be some sort of issue with awaiting the buffer.

[lovell]

It looks like you're writing the output to the input file so you'll need to use sharp.cache(false) to disable libvips' filesystem cache. I guess the latest sharp+libvips is just slightly faster enough that it's started to expose this existing race condition in your code.

In addition, the failOnError option was deprecated in v0.30.4 and you'll want to use the more fine-grained failOn instead.

[rdyar]

awesome, that seems to have worked. Thank you so much!
And yes, we are overwriting the input file. I saw the failOnError was deprecated, will fix that too.

I added sharp.cache(false) like:

async function stripRotation(filePath) {
  console.log(`Processing inside strip ${filePath}`);
  sharp.cache(false);
  const mainImage = sharp(filePath, {
    limitInputPixels: false,
    failOnError: false,
  });
  mainImage.jpeg({ quality: 98, chromaSubsampling: "4:4:4" });
  return mainImage.toBuffer();
}