Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WebAssembly build #3522

Closed
wants to merge 25 commits into from
Closed

WebAssembly build #3522

wants to merge 25 commits into from

Commits on Jan 9, 2023

  1. WebAssembly build

    This is a WebAssembly build of sharp usable in Node.js and Stackblitz environments.
    
    There is quite a lot going on here to make it work, so I won't list all the implementation details I had to make it in the PR description, but instead would ask you to refer to the comments which hopefully explains the individual hacks, and feel free to ask any questions in the review.
    
    The build script itself piggy-backs on the excellent work done by @kleisauke in [wasm-vips](https://github.com/kleisauke/wasm-vips) with some modifications to use it only for building libvips rather than custom bindings.
    
    One thing I'll emphasise is that this is a build made specifically for Node.js / Stackblitz with the intention of being 1:1 API-compatible - among other things, it means that Wasm instantiation is intentionally synchronous, and that it will use the native filesystem via Node.js raw filesystem. I had / have a
    separate branch with an almost working browser build of sharp, but that requires some API changes to be usable in a browser without locking the main thread and with accepting e.g. `File` object instead of using virtual filesystem, so for now keeping it out of scope.
    
    The concurrency is currently limited to a fixed-size threadpool. While I made it possible to create threads on-demand in recent versions of Emscripten, there are still some issues and bugs when trying to use it with internal libvips threadpool, so for now keeping a fixed-size threadpool is a safer and time-tested option. Among other things, this will run only one concurrent sharp operation at a time. Also, in Stackblitz environment libvips will be limited to only 2 threads to keep memory usage under control - this is done because libvips already always needs +3 extra threads, and async emnapi operation will need yet another +1 thread per async operation, so the number of Workers quickly adds up.
    
    Additionally, some formats and operations - namely, SVG, DZI and text operations - are currently unsupported just like they're in wasm-vips. Text (both on its own and in SVG) is notoriously difficult due to lots of questions around font loading (Local Font Access API, remote fonts, etc.), but other formats might come in
    time. For now, though, this Wasm build should already cover most common use-cases.
    
    Finally, for now I committed the build script together with prebuilt binaries as part of the PR, as it made testing easiest, but I'd ask the maintainers to integrate it properly into their "prebuilt addon" system somehow - I can't do that from my end, as I neither know how it works nor have access to the storage.
    RReverser committed Jan 9, 2023
    Configuration menu
    Copy the full SHA
    65e504a View commit details
    Browse the repository at this point in the history
  2. Simple CI test integration for Wasm

    Intentionally skipping coverage here, as Wasm doesn't cover everything.
    RReverser committed Jan 9, 2023
    Configuration menu
    Copy the full SHA
    b9e340c View commit details
    Browse the repository at this point in the history

Commits on Jan 10, 2023

  1. Configuration menu
    Copy the full SHA
    fd821e8 View commit details
    Browse the repository at this point in the history

Commits on Jan 11, 2023

  1. Upgrade wasm-vips

    RReverser committed Jan 11, 2023
    Configuration menu
    Copy the full SHA
    14bc120 View commit details
    Browse the repository at this point in the history

Commits on Jan 12, 2023

  1. Remove Embind

    Seems no longer necessary.
    RReverser committed Jan 12, 2023
    Configuration menu
    Copy the full SHA
    5185b7d View commit details
    Browse the repository at this point in the history

Commits on Feb 9, 2023

  1. Build with newer Emnapi

    RReverser committed Feb 9, 2023
    Configuration menu
    Copy the full SHA
    295d478 View commit details
    Browse the repository at this point in the history

Commits on Feb 14, 2023

  1. Upgrade Emnapi

    toyobayashi committed Feb 14, 2023
    Configuration menu
    Copy the full SHA
    77c6ced View commit details
    Browse the repository at this point in the history
  2. Commit wasm

    toyobayashi committed Feb 14, 2023
    Configuration menu
    Copy the full SHA
    3671432 View commit details
    Browse the repository at this point in the history

Commits on Feb 17, 2023

  1. Rebuild

    RReverser committed Feb 17, 2023
    Configuration menu
    Copy the full SHA
    1c9356d View commit details
    Browse the repository at this point in the history
  2. Lint fixup

    RReverser committed Feb 17, 2023
    Configuration menu
    Copy the full SHA
    4e4eab1 View commit details
    Browse the repository at this point in the history
  3. Upgrade wasm-vips

    RReverser committed Feb 17, 2023
    Configuration menu
    Copy the full SHA
    707341d View commit details
    Browse the repository at this point in the history
  4. Make tests parallel again

    RReverser committed Feb 17, 2023
    Configuration menu
    Copy the full SHA
    af35596 View commit details
    Browse the repository at this point in the history
  5. Remove manual async patching

    Upstream now does .ref/.unref automatically.
    RReverser committed Feb 17, 2023
    Configuration menu
    Copy the full SHA
    d5fe5e8 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    9e81ee7 View commit details
    Browse the repository at this point in the history
  7. Revert few more changes

    RReverser committed Feb 17, 2023
    Configuration menu
    Copy the full SHA
    514198f View commit details
    Browse the repository at this point in the history

Commits on Aug 5, 2023

  1. Configuration menu
    Copy the full SHA
    292d2a3 View commit details
    Browse the repository at this point in the history
  2. Update build flags

    RReverser committed Aug 5, 2023
    Configuration menu
    Copy the full SHA
    32a6c07 View commit details
    Browse the repository at this point in the history
  3. Update dep & rebuild

    RReverser committed Aug 5, 2023
    Configuration menu
    Copy the full SHA
    574f7d9 View commit details
    Browse the repository at this point in the history
  4. Rebuild

    RReverser committed Aug 5, 2023
    Configuration menu
    Copy the full SHA
    fce6988 View commit details
    Browse the repository at this point in the history
  5. Another attempt to fix npm install

    Little tricky but now this should work both for rebuilds and for normal installation as a dependency.
    RReverser committed Aug 5, 2023
    Configuration menu
    Copy the full SHA
    294edf3 View commit details
    Browse the repository at this point in the history
  6. One more fixup

    RReverser committed Aug 5, 2023
    Configuration menu
    Copy the full SHA
    06fa41f View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    e110f06 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    8642459 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    a840ba4 View commit details
    Browse the repository at this point in the history
  10. Fix format

    RReverser committed Aug 5, 2023
    Configuration menu
    Copy the full SHA
    aecf736 View commit details
    Browse the repository at this point in the history