Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WebAssembly build #3522

Closed
wants to merge 25 commits into from
Closed

WebAssembly build #3522

wants to merge 25 commits into from

Conversation

RReverser
Copy link
Contributor

@RReverser RReverser commented Jan 9, 2023

This adds a WebAssembly build of sharp usable in Node.js and Stackblitz environments.

There is quite a lot going on here to make it work, so I won't list all the implementation details I had to make it in the PR description, but instead would ask you to refer to the comments which hopefully explains the individual hacks; and, of course, feel free to ask any questions in the review. Some things were already upstreamed to Emscripten and emnapi, and others probably will be over time, hopefully simplifying the porting process for everyone, but for now some hacks remain.

The build script itself piggy-backs on the excellent work done by @kleisauke in wasm-vips with some modifications to use it only for building libvips rather than custom bindings.

One thing I'll emphasise is that this is a build made specifically for Node.js / Stackblitz with the intention of being 1:1 API-compatible - among other things, it means that Wasm instantiation is intentionally synchronous, and that it will use the native filesystem via Node.js raw filesystem. I had / have a
separate branch with an almost working browser build of sharp, but that requires some API changes to be usable in a browser without locking the main thread and with accepting e.g. File object instead of using virtual filesystem, so for now keeping it out of scope.

The concurrency is currently limited to a fixed-size threadpool. While I made it possible to create threads on-demand in recent versions of Emscripten, there are still some issues and bugs when trying to use it with internal libvips threadpool, so for now keeping a fixed-size threadpool is a safer and time-tested option. Among other things, this will run only one concurrent sharp operation at a time. Also, in Stackblitz environment libvips will be limited to only 2 threads to keep memory usage under control - this is done because libvips already always needs +3 extra threads, and async emnapi operation will need yet another +1 thread per async operation, so the number of Workers quickly adds up.

Additionally, some formats and operations - namely, SVG, DZI and text operations - are currently unsupported just like they're in wasm-vips. Text (both on its own and in SVG) is notoriously difficult due to lots of questions around font loading (Local Font Access API, remote fonts, etc.), but other formats might come in time. For now, though, this Wasm build should already cover most common use-cases. I did have to modify some tests to use more widely supported formats as inputs though, as well as sprinke some conditional skips for npm test --arch=wasm32 to work.

Finally, for now I committed the build script together with prebuilt binaries as part of the PR, as it made testing easiest, but I'd ask the maintainers to integrate it properly into their "prebuilt addon" system somehow - I can't do that from my end, as I neither know how it works nor have access to the storage.

Oh, and for the curious here are the native x64 vs Wasm benchmarks executed with various concurrency options (raw CSV here):

Chart

sharp-benchmarks

As you can see, Wasm performs at around native speed in some benchmarks, but is ~2x slower where SIMD is required - which is expected, given that libvips heavily relies on runtime SIMD code generation, which is not yet supported for Wasm.

Fixes #3336.

@RReverser RReverser force-pushed the node-wasm-squash branch 2 times, most recently from a55afb0 to 82c0159 Compare January 9, 2023 18:29
binding.gyp Outdated Show resolved Hide resolved
binding.gyp Show resolved Hide resolved
This is a WebAssembly build of sharp usable in Node.js and Stackblitz environments.

There is quite a lot going on here to make it work, so I won't list all the implementation details I had to make it in the PR description, but instead would ask you to refer to the comments which hopefully explains the individual hacks, and feel free to ask any questions in the review.

The build script itself piggy-backs on the excellent work done by @kleisauke in [wasm-vips](https://github.com/kleisauke/wasm-vips) with some modifications to use it only for building libvips rather than custom bindings.

One thing I'll emphasise is that this is a build made specifically for Node.js / Stackblitz with the intention of being 1:1 API-compatible - among other things, it means that Wasm instantiation is intentionally synchronous, and that it will use the native filesystem via Node.js raw filesystem. I had / have a
separate branch with an almost working browser build of sharp, but that requires some API changes to be usable in a browser without locking the main thread and with accepting e.g. `File` object instead of using virtual filesystem, so for now keeping it out of scope.

The concurrency is currently limited to a fixed-size threadpool. While I made it possible to create threads on-demand in recent versions of Emscripten, there are still some issues and bugs when trying to use it with internal libvips threadpool, so for now keeping a fixed-size threadpool is a safer and time-tested option. Among other things, this will run only one concurrent sharp operation at a time. Also, in Stackblitz environment libvips will be limited to only 2 threads to keep memory usage under control - this is done because libvips already always needs +3 extra threads, and async emnapi operation will need yet another +1 thread per async operation, so the number of Workers quickly adds up.

Additionally, some formats and operations - namely, SVG, DZI and text operations - are currently unsupported just like they're in wasm-vips. Text (both on its own and in SVG) is notoriously difficult due to lots of questions around font loading (Local Font Access API, remote fonts, etc.), but other formats might come in
time. For now, though, this Wasm build should already cover most common use-cases.

Finally, for now I committed the build script together with prebuilt binaries as part of the PR, as it made testing easiest, but I'd ask the maintainers to integrate it properly into their "prebuilt addon" system somehow - I can't do that from my end, as I neither know how it works nor have access to the storage.
Intentionally skipping coverage here, as Wasm doesn't cover everything.
@RReverser
Copy link
Contributor Author

So there's one flaky test that I'm rarely seeing locally, but currently failed on CI, and whenever I saw it, it always showed number 52:

  1) Utilities
       Cache
         Can be disabled:

      AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:

52 !== 0

      + expected - actual

      -52
      +0
      
      at Context.<anonymous> (test/unit/util.js:11:14)
      at process.processImmediate (node:internal/timers:471:21)

It seems to suggest there's a leak of 52 MB somewhere in the Wasm build (at least I haven't seen it failing on native builds yet), but no idea where :/

@RReverser

This comment was marked as outdated.

@RReverser

This comment was marked as outdated.

@RReverser
Copy link
Contributor Author

Ah ok I can reproduce this consistently locally when I disable "parallel": true, but also only on Wasm.

@RReverser
Copy link
Contributor Author

Narrowed down to failOn.js and timeout.js leaving some leaks (each individually) that the memory check seems to pick up.

Interesting that both tests are specifically for failures.

@RReverser
Copy link
Contributor Author

Went through tests in those suites by uncommenting one by one and looks like only these 3 tests cause the detected memory leak:

  • returns errors to callback for truncated JPEG
  • rejects promises for truncated JPEG
  • Will timeout after 1s when performing slow blur operation

It's interesting that all three 1) use JPEG input, same tests for e.g. PNG are fine and 2) return an error.

It increasingly looks like a legitimate leak in... mozjpeg? vips JPEG module?... but only when built on Wasm and only somewhere in error handling path.

@RReverser

This comment was marked as outdated.

@RReverser
Copy link
Contributor Author

If I'm reading VIPS_LEAK logs correctly, it seems to confirm that JPEG image is the one leaked (although there's a bunch of other objects here):

44 objects alive:
0) VipsImage (0xee0040) 32 bytes, count=2
VipsImage (image), image class, 2725x2225 uchar, 3 bands, srgb, jpegload
1) VipsSequential (0x35a85d8), count=1
VipsSequential (sequential), check sequential access, sequential in=((VipsImage*) 0x358cd40) out=((VipsImage*) 0x35a8820) tile-height=8 -
2) VipsEmbed (0xed5270), count=1
VipsEmbed (embed), embed an image in a larger image, embed in=((VipsImage*) 0xec8b10) out=((VipsImage*) 0xed5680) x=179 y=0 width=3083 height=2225 extend=((VipsExtend) VIPS_EXTEND_COPY) -
3) VipsImage (0xede6e8), count=1
VipsImage (image), image class, 2725x2583 uchar, 3 bands, srgb, jpegload
4) VipsImage (0xed7040) 32 bytes, count=2
VipsImage (image), image class, 2725x2225 uchar, 3 bands, srgb, jpegload
5) VipsSource (0xe92948), count=3
VipsSource (source), input source, 
6) VipsImage (0xede888), count=1
VipsImage (image), image class, 1x359 double, 1 band, matrix
7) VipsImage (0xed5680), count=1
VipsImage (image), image class, 3083x2225 uchar, 3 bands, srgb, jpegload
8) VipsImage (0xed5820), count=1
VipsImage (image), image class, 359x1 double, 1 band, matrix
9) VipsLineCache (0xed7ca8), count=1
VipsLineCache (linecache), cache an image as a set of lines, linecache in=((VipsImage*) 0x358cd40) out=((VipsImage*) 0x35a95f8) tile-height=8 access=((VipsAccess) VIPS_ACCESS_SEQUENTIAL) -
10) VipsImage (0xe92b68) 824 bytes, count=4
VipsImage (image), image class, 2725x2225 uchar, 3 bands, srgb, jpegload
11) VipsExtractArea (0x35a9f78), count=1
VipsExtractArea (extract_area), extract an area from an image, extract_area input=((VipsImage*) 0x35a8820) out=((VipsImage*) 0x35aa210) left=0 top=0 width=2725 height=2225 -
12) VipsImage (0xec8b10), count=3
VipsImage (image), image class, 2725x2225 uchar, 3 bands, srgb, jpegload
13) VipsImage (0xed9d50), count=2
VipsImage (image), image class, 2725x2225 uchar, 3 bands, srgb, jpegload
14) VipsImage (0xeb5ab0), count=2
VipsImage (image), image class, 359x1 double, 1 band, multiband
15) VipsImage (0xeb4480), count=2
VipsImage (image), image class, 1x359 double, 1 band, multiband
16) VipsForeignLoadJpegFile (0xe92600), count=1
VipsForeignLoadJpegFile (jpegload), load jpeg from file (.jpg, .jpeg, .jpe), priority=50, is_a, get_flags, get_flags_filename, header, load, jpegload filename="/home/rreverser/sharp/test/fixtures/2569067123_aca715a2ee_o.jpg" out=((VipsImage*) 0xe92b68) flags=((VipsForeignFlags) VIPS_FOREIGN_SEQUENTIAL) access=((VipsAccess) VIPS_ACCESS_RANDOM) fail-on=((VipsFailOn) VIPS_FAIL_ON_WARNING) -
17) VipsImage (0xe9e228), count=2
VipsImage (image), image class, 2725x2225 uchar, 3 bands, srgb, jpegload
18) VipsCopy (0xeda298), count=1
VipsCopy (copy), copy an image, copy in=((VipsImage*) 0xec1200) out=((VipsImage*) 0xeda608) -
19) VipsCopy (0xeac9f8), count=1
VipsCopy (copy), copy an image, copy in=((VipsImage*) 0xeace08) out=((VipsImage*) 0xeb4480) -
20) VipsGaussmat (0xe9c9a0), count=1
VipsGaussmat (gaussmat), make a gaussian image, gaussmat out=((VipsImage*) 0xe9cc10) sigma=100.000000 min-ampl=0.200000 separable=TRUE precision=((VipsPrecision) VIPS_PRECISION_INTEGER) -
21) VipsRot (0xe32440), count=1
VipsRot (rot), rotate an image, rot in=((VipsImage*) 0xe9f368) out=((VipsImage*) 0xeace08) angle=((VipsAngle) VIPS_ANGLE_D90) -
22) VipsImage (0xed8b48), count=2
VipsImage (image), image class, 1x359 double, 1 band, multiband
23) VipsImage (0xe9cc10), count=2
VipsImage (image), image class, 359x1 double, 1 band, multiband
24) VipsImage (0xeace08), count=2
VipsImage (image), image class, 1x359 double, 1 band, multiband
25) VipsImage (0xec9f98), count=1
VipsImage (image), image class, 359x1 double, 1 band, multiband
26) VipsImage (0x358a908) 824 bytes, count=1
VipsImage (image), image class, 2725x2225 uchar, 3 bands, srgb
27) VipsImage (0x35a95f8), count=1
VipsImage (image), image class, 2725x2225 uchar, 3 bands, srgb
28) VipsImage (0xeda608), count=3
VipsImage (image), image class, 2725x2225 uchar, 3 bands, srgb, jpegload
29) VipsImage (0xedbab0), count=1
VipsImage (image), image class, 1x359 double, 1 band, multiband
30) VipsImage (0x35a8820), count=2
VipsImage (image), image class, 2725x2225 uchar, 3 bands, srgb
31) VipsImage (0x358cd40), count=3
VipsImage (image), image class, 2725x2225 uchar, 3 bands, srgb
32) VipsConvsep (0xe9def8), count=1
VipsConvsep (convsep), seperable convolution operation, convsep in=((VipsImage*) 0xe92b68) out=((VipsImage*) 0xe9e228) mask=((VipsImage*) 0xe9cc10) precision=((VipsPrecision) VIPS_PRECISION_INTEGER) -
33) VipsCopy (0xebb480), count=1
VipsCopy (copy), copy an image, copy in=((VipsImage*) 0xe92b68) out=((VipsImage*) 0xec8b10) -
34) VipsImage (0x35aa210), count=1
VipsImage (image), image class, 2725x2225 uchar, 3 bands, srgb
35) VipsConv (0xeb4bc8), count=1
VipsConv (conv), convolution operation, conv in=((VipsImage*) 0xe92b68) out=((VipsImage*) 0xec1200) mask=((VipsImage*) 0xe9f368) precision=((VipsPrecision) VIPS_PRECISION_INTEGER) layers=5 cluster=1 -
36) VipsConvi (0xec9270) 2872 bytes, count=1
VipsConvi (convi), int convolution operation, convi in=((VipsImage*) 0xec8b10) out=((VipsImage*) 0xed7040) mask=((VipsImage*) 0xeb5ab0) -
37) VipsConv (0xed7a40), count=1
VipsConv (conv), convolution operation, conv in=((VipsImage*) 0xec1200) out=((VipsImage*) 0xed9d50) mask=((VipsImage*) 0xeb4480) precision=((VipsPrecision) VIPS_PRECISION_INTEGER) layers=5 cluster=1 -
38) VipsImage (0xe9f368), count=3
VipsImage (image), image class, 359x1 double, 1 band, multiband
39) VipsConvi (0xedad88) 2872 bytes, count=1
VipsConvi (convi), int convolution operation, convi in=((VipsImage*) 0xeda608) out=((VipsImage*) 0xee0040) mask=((VipsImage*) 0xed8b48) -
40) VipsImage (0xec1200), count=3
VipsImage (image), image class, 2725x2225 uchar, 3 bands, srgb, jpegload
41) VipsGaussblur (0xe8b340), count=1
VipsGaussblur (gaussblur), gaussian blur, gaussblur in=((VipsImage*) 0xe92b68) out=((VipsImage*) 0xee0f50) sigma=100.000000 -
42) VipsEmbed (0xede2d8), count=1
VipsEmbed (embed), embed an image in a larger image, embed in=((VipsImage*) 0xeda608) out=((VipsImage*) 0xede6e8) x=0 y=179 width=2725 height=2583 extend=((VipsExtend) VIPS_EXTEND_COPY) -
43) VipsImage (0xee0f50) 4 bytes, count=1
VipsImage (image), image class, 2725x2225 uchar, 3 bands, srgb, jpegload
5 VipsArea alive
        0x35913a8 count = 6, bytes = 5
        0xede3e8 count = 1, bytes = 8
        0xed5380 count = 1, bytes = 8
        0xe9c688 count = 10, bytes = 5
        0xe93130 count = 11, bytes = 8
memory: 9 allocations, 18212495 bytes
files: 0 open
memory: high-water mark 25.13 MB

@kleisauke This is probably a leak that affects wasm-vips itself too, rather than specific to this PR.

@RReverser
Copy link
Contributor Author

Resetted the branch to exclude my gc() experiments as that wasn't the issue and there is an actual leak. Will probably leave to someone more familiar with libvips to investigate why mozjpeg behaves differently when used instead of libjpeg-turbo.

@RReverser

This comment was marked as outdated.

@RReverser
Copy link
Contributor Author

Can't help but feel some sort of irony at the coincidence, because the last time I wrote an article on debugging memory leaks in Wasm, it also involved mozjpeg 😅 https://web.dev/webassembly-memory-debugging/

I don't think it's the same issue though - as far as I can tell, libvips correctly calls jpeg_destroy_compress, and other bugs were specifically related to the way Squoosh did bindings.

@lovell
Copy link
Owner

lovell commented Jan 10, 2023

Wow, thank you very much Ingvar, this looks like a great start in getting sharp compiled to WASM. дякую!

Over the next few days I'd like to focus on getting the new release out (major libvips upgrade, TypeScript definitions moving "in-house", etc.), then will come back and take a closer look at your PR. I will have questions :)

@RReverser
Copy link
Contributor Author

I will have questions :)

No-no, just click that merge button 🙈

@RReverser
Copy link
Contributor Author

this looks like a great start in getting sharp compiled to WASM

I'm curious - why start? Do you mean in terms of missing formats, or browser support or something else?

дякую!

💗

@d3lm
Copy link

d3lm commented Jan 10, 2023

Fantastic work @RReverser, really 👏 👏 👏 I can't wait for this to be integrated upstream. This will be a game changer for the Web and also a great case study for Wasm and the Node.js ecosystem.

@RReverser
Copy link
Contributor Author

RReverser commented Jan 10, 2023

Okay @kleisauke and me found the reason for the leak (in case of wasm-vips, 2 separate reasons 😅).

I committed the rebuilt JS+Wasm binaries to confirm it on the CI as well - which passed on wasm32 now (x64 seems a flake) - but I'll wait on him to update wasm-vips repo so that I could update the source hash in the build script.

UPD: done.

@toyobayashi
Copy link

Nice to see emnapi is used in so great project!

I'm a bit curious why emnapi needs embind, is there any problem without linking embind?

https://github.com/RReverser/sharp-fork/blob/14bc12012f4c3408300331f6415cc994b0df73f6/wasm-scripts/common.gypi#L45-L46

@RReverser
Copy link
Contributor Author

Nice to see emnapi is used in so great project!

I'm a bit curious why emnapi needs embind, is there any problem without linking embind?

RReverser/sharp-fork@14bc120/wasm-scripts/common.gypi#L45-L46

Uhh good question 😅 I think it was necessary before? I'm not sure, but there was a reason I added it back when I started...

Happy to remove now.

Seems no longer necessary.
@birkskyum
Copy link

birkskyum commented Aug 24, 2023

I doubt this install overhaul will be finished in 2023, so we need a temporary solution here. Ideally this would go into sharp repo, but publishing sharp-wasm to npm until the refactor is over would be very helpful. The method to install from repo works in some cases, but I find it doesn't work well in CI, deno and bun.

@styfle
Copy link

styfle commented Aug 28, 2023

@kleisauke Thanks for the code snippet, that was helpful!

I ran the Wasm build against all the Next.js tests and all of them pass.

However, I found that AVIF was extremely slow. It actually performed worse than Squoosh which is also Wasm.

Running benchmarks on darwin (arm64) with 10 images...

Benchmark f=webp, w=2048, q=75
      sharp-native: avg 0121ms, p90 0104ms, max 0271ms, min 0007ms
        sharp-wasm: avg 0206ms, p90 0161ms, max 0612ms, min 0015ms
           squoosh: avg 0411ms, p90 0308ms, max 1338ms, min 0015ms

Benchmark f=webp, w=1080, q=75
      sharp-native: avg 0079ms, p90 0070ms, max 0160ms, min 0007ms
        sharp-wasm: avg 0105ms, p90 0091ms, max 0233ms, min 0012ms
           squoosh: avg 0282ms, p90 0242ms, max 0640ms, min 0017ms

Benchmark f=webp, w=256, q=50
      sharp-native: avg 0020ms, p90 0019ms, max 0032ms, min 0004ms
        sharp-wasm: avg 0035ms, p90 0032ms, max 0061ms, min 0007ms
           squoosh: avg 0107ms, p90 0093ms, max 0227ms, min 0017ms

Benchmark f=avif, w=2048, q=75
      sharp-native: avg 0611ms, p90 0473ms, max 1857ms, min 0034ms
        sharp-wasm: avg 5703ms, p90 4444ms, max 17030ms, min 0215ms
           squoosh: avg 2131ms, p90 1535ms, max 7498ms, min 0082ms

Benchmark f=avif, w=1080, q=75
      sharp-native: avg 0367ms, p90 0272ms, max 1219ms, min 0033ms
        sharp-wasm: avg 2550ms, p90 1842ms, max 8925ms, min 0213ms
           squoosh: avg 1314ms, p90 0959ms, max 4516ms, min 0081ms

Benchmark f=avif, w=256, q=50
      sharp-native: avg 0056ms, p90 0047ms, max 0137ms, min 0020ms
        sharp-wasm: avg 0188ms, p90 0156ms, max 0482ms, min 0078ms
           squoosh: avg 0189ms, p90 0162ms, max 0428ms, min 0050ms
Benchmark complete.

The one that stands out is Benchmark f=avif, w=2048, q=75 because sharp wasm takes over 2x longer than squoosh, and this is tested against 10 different images on Node.js 18.17.1

@RReverser This makes me think there might be a bug. The code running in this case would be:

const sharp = require(`sharp`);
const transformer = sharp(bufferInput)
transformer.resize(2048, undefined, { withoutEnlargement: true })
transformer.avif({ quality: 60, chromaSubsampling: '4:2:0' })
const result = await transformer.toBuffer()

@RReverser
Copy link
Contributor Author

RReverser commented Sep 2, 2023

@RReverser This makes me think there might be a bug.

I don't think it's a bug. Sharp (or, rather, libvips) and Squoosh use different libraries for AVIF compression, as well as handle multithreading in very different ways, so differences are expected. You can bring it up with libvips if you can reproduce it natively too, but that's out of scope of this PR.

@lovell
Copy link
Owner

lovell commented Oct 16, 2023

Now that #3750 is in a good place, I've finally made some time to start to look at this PR and understand how best to integrate it. Thank you for your patience.

The prebuilt libvips binaries provided for use with sharp are, where possible, a single shared library containing libvips and all of its dependencies statically linked.

The prebuilt sharp binaries are a shared library with a dependency on the above shared library.

$ ldd node_modules/@img/sharp-linux-x64/lib/sharp-linux-x64.node 
	linux-vdso.so.1 (0x00007ffcf3cc1000)
	libvips-cpp.so.42 => node_modules/@img/sharp-linux-x64/lib/../../sharp-libvips-linux-x64/lib/libvips-cpp.so.42 (0x00007fc2d0000000)
	libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fc2cfc00000)
	...

This is done partly to help clarify licensing. User code has a shared library dependency on Apache 2 sharp, which has a shared library dependency on LGPLv2+ libvips. Mixing sharp and libvips (and other LGPLv2+/v3) code in the same shared library could mean parts of sharp itself falling under the terms of LGPLv3, which might introduce a very small risk of GPL exposure that some would not be willing to take.

What options are available, if any, for keeping these shared libraries separate in WASM? For example, could we publish vips-cpp.wasm and sharp.wasm in separate npm packages? It looks like there is some support for dynamic linking via SIDE_MODULE but I'm unsure of the runtime mechanics (e.g. no rpath).

@RReverser
Copy link
Contributor Author

RReverser commented Oct 16, 2023

Now that #3750 is in a good place, I've finally made some time to start to look at this PR and understand how best to integrate it. Thank you for your patience.

Yay! Glad to see progress!

For example, could we publish vips-cpp.wasm and sharp.wasm in separate npm packages? It looks like there is some support for dynamic linking via SIDE_MODULE but I'm unsure of the runtime mechanics (e.g. no rpath).

Yeah there is no official dynamic linking in Wasm (yet and likely not soon); there is this Emscripten's side module mechanism but it tends to be problematic both in terms of bugs and performance overhead (as now you need to have different Wasm modules, individually compiled by the engine, communicating to each other through separate pieces of JS), so it's normally recommended to avoid it in favour of static linking.

If it's not too much of a concern, I'd rather recommend sticking to static linking as well. I assume the different license would only apply to users of the Wasm npm package if Wasm binaries aren't going to be actually checked into the repo?

@RReverser
Copy link
Contributor Author

For the sake of setting clear expectations, I probably should also say that it's been a while and meanwhile I've switched and am working on other projects. I'd still want to see this landed and will try my best to respond here to any comments and minor suggestions, but can't promise time commitment for doing actual work on further integration.

@lovell
Copy link
Owner

lovell commented Oct 19, 2023

@RReverser Thanks Ingvar, I've spent the last couple of days immersing myself in the world of WebAssembly and have come out the other side with a much better understanding, or at least with enough experience to be dangerous ;)

Your advice on linking is really useful, I agree static is the way to go. As a result the use of wasm will be opt-in e.g. "add package X to your dependencies" and will require suitable licensing notices.

If you hadn't seen, the forthcoming libvips 8.15.0 will include support for SIMD via highway, which means (greater) SIMD support for WebAssembly too.

@toyobayashi Loving your work on emnapi, it's brilliant, 谢谢!

@RReverser
Copy link
Contributor Author

If you hadn't seen, the forthcoming libvips 8.15.0 will include support for SIMD via highway, which means (greater) SIMD support for WebAssembly too.

That's awesome, should be easy to make use of it when it's out.

@toyobayashi
Copy link

Loving your work on emnapi, it's brilliant, 谢谢!

@lovell Thank you! Great to see sharp wasm succeed. Also great thanks to @RReverser for his many valuable suggestions and contributions to emnapi on technical details during porting sharp, without Ingvar's help, emnapi would not be as complete as it is today.

@lovell
Copy link
Owner

lovell commented Nov 6, 2023

I've split this change into two separate (draft) PRs and updated to fit with the new installation package structure.

There will be a prerelease soon should people wish to help test these - please see #3750 for updates.

@lovell
Copy link
Owner

lovell commented Nov 10, 2023

This PR has landed as lovell/sharp-libvips@e219aea and a8f68ba and can now be tested via the instructions in #3750 (comment). It will be included as part of sharp v0.33.0.

дякую @RReverser for all your work on this, 谢谢 @toyobayashi for emnapi and, as always, dankjewel @kleisauke for wasm-vips where this all began.

For StackBlitz support please see stackblitz/webcontainer-core#1236

@lovell lovell closed this Nov 10, 2023
@lovell lovell added this to the v0.33.0 milestone Nov 10, 2023
@RReverser
Copy link
Contributor Author

Amazing, thanks @lovell!

@d3lm
Copy link

d3lm commented Nov 10, 2023

This is huge! Awesome work everyone and thanks a lot again @RReverser for working on this with us. Thanks @lovell for getting this merged! 🙏

@odgey
Copy link

odgey commented Nov 21, 2023

Wow this is very interesting, I came across this thread researching for the next itteration of our web based email designer. Were currently using Sharp server-side using an XHR request to sanity check and resize the dimensions of the image to a sensible size.

One of the things on the list was client side processing for this.

@RReverser it sounds like that's a separate project branch you have? being able to pass in a file object and manipulate in the browser would be perfect - are there any plans to push this at some point. May not be able to contribute with the code but can certainly help testing in a perfect use case

@serban-mihai
Copy link

I would also love a browser build of Sharp, being able to scale images and generate thumbnails on the client side before uploads would diminish stress over the remote architecture, cut costs, and increase speed!

@birkskyum
Copy link

I would also love a browser build of Sharp, being able to scale images and generate thumbnails on the client side before uploads would diminish stress over the remote architecture, cut costs, and increase speed!

That really would be awesome - I have some applications where sharp processing on the server becomes the perf bottleneck with many users active, and distributing that work to their clients would fix that.

@RReverser
Copy link
Contributor Author

RReverser commented Nov 29, 2023

@RReverser it sounds like that's a separate project branch you have? being able to pass in a file object and manipulate in the browser would be perfect - are there any plans to push this at some point. May not be able to contribute with the code but can certainly help testing in a perfect use case

Yeah I can probably rebase & push it somewhere now that this PR is done. Most has been done, but I won't be able to work on remaining issues further myself, maybe someone can fork and continue that way.

That said, I wonder if for browser and Cloudflare Workers usecases something like https://github.com/jamsinclair/jSquash would be better suited? (no offence @lovell)

It's a fork of Squoosh.app but as a library. It's a bit more modular, since it compiles those codecs individually, and, by avoiding libvips, it can also run in environments without Web Workers.

@odgey
Copy link

odgey commented Jan 4, 2024

@RReverser Thanks for the steer on JSquash I've had a good look at that and integrated the code but the performance is poor. I note someone else has recently raised that as an issue so it looks like the code around the codecs is some kind of bottle-neck. I've also experienced crashes on images that work on the core Squoosh app

Unpicking the react front end from squoosh is an option but it does not support animated GIF.

I know animated GIF in 2024!! but my use case is an email designer and email design is stuck in 1994 and animated GIF is still a "thing" and Sharp does a great job.

I can take a look at forking the project if it's rebased, never done anything with WASM so it may be a learning curve :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

WASM Support