Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

faster decoding #20

Merged
merged 25 commits into from
Aug 31, 2021
Merged

faster decoding #20

merged 25 commits into from
Aug 31, 2021

Conversation

jerch
Copy link
Owner

@jerch jerch commented Jun 23, 2021

The PR introduces a webassembly decoder for faster decoding. Speedup is 3-4x.

Also fixes #17.

TODO:

  • reconcile decoder interface
  • more test cases
  • more prebundles
  • set up gh actions as CI
  • fix example scripts
  • polish docs

- add DEFAULT_FOREGROUND color in Color.ts
- removed DimensionDecoder (now part of wasm code)
- removed SixelDecoderL2 (now part of wasm code)
- fix prolly undefined palette color in encoder
- wasm decoder:
  - changed to handle only one band at a time
  - static fixed memory with upper max band width
  - optimize with bulk-memory
  - added normal M1 for level 1 images or non truncating
  - cleanup emscripten install & wam build scripts
  - python wasmer example
  - wasm-simd proof of concept implementation
- moved WasmDecoder.ts to Decoder.ts
- Decoder.ts prepared as new main decoder
- convenient decode functions
- add new decoder to benchmarks
- cleanup imports/export in index.ts
- better build scripts:
  - remove the need for json imports
  - add UMD/ESM bundles for browser/webpack
@jerch
Copy link
Owner Author

jerch commented Aug 28, 2021

@PerBothner Just finished the rework of the decoder details and some of the bundling aspects. It is still not fully finished for release (tests missing, old decoder to be removed, docs cleanup). In case you want to test things early, you can do that as follows:

git clone git@github.com:jerch/node-sixel.git
cd node-sixel
git fetch origin pull/20/head:faster_decode
git checkout faster_decode
npm install

(This will take some time to install emscripten.)

If all went well, you'll find under /dist several browser bundles. I guess you are only interested in the decoder part, thus the decode.umd.js (umd build) or decode.esm.js (es6 module) should do.

Note that the decoder interface changed alot (mostly simplified, for now check in Decoder.ts).

On my machine these are the benchmark results:

   Context "./lib/index.benchmark.js"
      Context "testimage"
         Context "pixel transfer"
            Case "toPixelData - with fillColor" : 20 runs - average runtime: 1.48 ms
            Case "toPixelData - without fillColor" : 20 runs - average runtime: 0.87 ms
         Context "decode (DefaultDecoder)"
            Case "decode" : 20 runs - average runtime: 3.83 ms
            Case "decodeString" : 20 runs - average runtime: 4.11 ms
            Case "decode + pixel transfer" : 20 runs - average runtime: 3.09 ms
         Context "decode (WasmDecoder)"
            Case "decode" : 20 runs - average runtime: 0.76 ms
            Case "decodeString" : 20 runs - average runtime: 1.48 ms
         Context "encode"
            Case "sixelEncode" : 20 runs - average runtime: 21.57 ms
      Context "decode - testfiles (DefaultDecoder)"
         Case "test1_clean.sixel" : 20 runs - average runtime: 16.20 ms
         Case "test1_clean.sixel" : 20 runs - average throughput: 38.57 MB/s
         Case "test2_clean.sixel" : 20 runs - average runtime: 6.49 ms
         Case "test2_clean.sixel" : 20 runs - average throughput: 48.75 MB/s
         Case "sampsa_reencoded_clean.six" : 20 runs - average runtime: 15.76 ms
         Case "sampsa_reencoded_clean.six" : 20 runs - average throughput: 40.98 MB/s
         Case "FullHD 12bit noise" : 20 runs - average runtime: 224.61 ms
         Case "FullHD 12bit noise" : 20 runs - average throughput: 69.03 MB/s
      Context "decode - testfiles (WasmDecoder)"
         Case "test1_clean.sixel" : 20 runs - average runtime: 3.89 ms
         Case "test1_clean.sixel" : 20 runs - average throughput: 152.63 MB/s
         Case "test2_clean.sixel" : 20 runs - average runtime: 1.91 ms
         Case "test2_clean.sixel" : 20 runs - average throughput: 165.01 MB/s
         Case "sampsa_reencoded_clean.six" : 20 runs - average runtime: 4.47 ms
         Case "sampsa_reencoded_clean.six" : 20 runs - average throughput: 146.42 MB/s
         Case "FullHD 12bit noise" : 20 runs - average runtime: 48.53 ms
         Case "FullHD 12bit noise" : 20 runs - average throughput: 319.51 MB/s

(DefaultDecoder is the old one, WasmDecoder the new one.)
Speedup is around 3-4 times now. 😸

@jerch jerch merged commit 2c74b11 into master Aug 31, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

following CR or LF not respected in APPLY_PARAM
1 participant