Bundles are big

[elsehow]

I've been finding that bundles with cosmjs tend to be pretty big (600k+), much of which appears to be due to the dependency on libsodium.

Here's an example from a recent (next.js) build:

yarn why libsodium
yarn why v1.22.17
[1/4] Why do we have the module "libsodium"...?
[2/4] Initialising dependency graph...
[3/4] Finding dependency...
[4/4] Calculating file sizes...
=> Found "libsodium@0.7.9"
info Reasons this module exists
   - "@cosmjs#cosmwasm-stargate#@cosmjs#crypto#libsodium-wrappers" depends on it
   - Hoisted from "@cosmjs#cosmwasm-stargate#@cosmjs#crypto#libsodium-wrappers#libsodium"
info Disk size without dependencies: "508KB"
info Disk size with unique dependencies: "508KB"
info Disk size with transitive dependencies: "508KB"
info Number of shared dependencies: 0
Done in 0.52s.

It seems cosmjs's dependency on libsodium comes from this import.

I wonder: is it possible to optimize our use of libsodium to minimize its footprint, or at least make it more amenable to optimization by static analyzers?

[webmaster128]

Thank you for bringing this up. This issue came up at Confio as well and we are happy to improve the situation. However, it will not be easy.

Improvement potential:

• libsodium.js ships with a JavaScript fallsback of all its code (asm.js). We don't need that and can assume Wasm in all supported environments.
• libsodium.js ships much more functionality than we need

We could try a custom build of libsodium.js, but that takes a lot of effort. What I think makes more sense is replacing the functionality we actually need:

• Argon2 could be taken from Rust and compiled to Wasm (https://crates.io/crates/argon2) if we make a JavaScript/wasm-bindgen wrapper
• Ed25519 should be taken from zebra (Switch Ed25519 implementation to zip215 compliant library #584)
• xchacha20poly1305-ietf is probably also available somewhere in Rust

Last time I tried I was in big troubly trying to embed Wasm in a library using wasm-bindgen + webpack4 (and some other environment constraints I forget). But maybe this part got easier over the months.

[elsehow]

Thanks for the detailed reply. I'm certainly on the side of avoiding a custom libsodium build - that would be an ongoing headache.

xchacha20poly1305-ietf is probably also available somewhere in Rust

Absolutely, a version is maintained by RustCrypto: https://crates.io/crates/chacha20poly1305

I have not kept up with Wasm/JS integration, but I'm more than happy to do a bit of research.

That said, has anyone tried creating a bundled build with the Google Closure compiler? It's certainly the simplest thing to try first, and I wouldn't be shocked if it successfully removes a good deal of libsodium cruft.

[webmaster128]

In #907 I started looking into Argon2. This is probably the simplest but most speed critical one. It's cool to see this compile to only 44KB of (uncompressed) Wasm. But unfortunately it's slower than what we have right now.

That said, has anyone tried creating a bundled build with the Google Closure compiler?

Never used it. But also don't have hopes for libsodium.js, because this is big due to Wasm and asm.js which are already compiler outputs.

[elsehow]

I see, that's a significant performance hit. In that case, I'm inclined to say that a custom build of libsodium.js may not be the worst thing. We really just need to tweak this one flag in the Emscripten build:

https://github.com/jedisct1/libsodium/blob/95673e5b51e750c5eee1aecd935cbfc5791d741b/dist-build/emscripten.sh#L4

[webmaster128]

Do you want to give it a shot? Maybe we can even get 3 builds, one for Ed25519, xchacha20poly1305-ietf, Argon2. Then we have the flexibility to replace them and optimizers might find it easier to remove unused stuff. E.g. for most applications, Ed25519 will be unused at the moment.

[elsehow]

That's a great idea. It could be of broad use (beyond cosmsj) to have these methods broken out. I'll have a go at it!

[elsehow]

Update: separating out the packages doesn't save nearly as much space as I expected. xchacha20poly1305ietf and argon2 alone are about 400kb.

The underlying causes of this bloat appear to be well-described in this issue: jedisct1/libsodium.js#263 One issue is that legacy asm.js and wasm are combined in the JS builds. This helps support older IE (and other browsers that don't support wasm), but increases the size of the library dramatically. Another issue is that the lack of ES6 modules in the build prevent tree shaking.

I think we're better off solving these two problems within libsodium.js than we are producing a custom build for cosmjs. After these changes are done, we can always do some tree shaking in cosmjs or downstream of it to reduce bundle size.

[gavindoughtie]

I’m going to say right here that chasing IE11 support is a waste of everyone’s energy. Google deprecated it; what’s the incentive to keep it going?

[webmaster128]

Neither IE11 nor pre-Chromium Edge have ever been supported by CosmJS (see https://github.com/cosmos/cosmjs#supported-js-environments). However, our dependencies may have different strategies. Also there are reports that executing Wasm in a browser extension is a problem. Not sure what the exact problem is though.

[webmaster128]

@elsehow could you let me know how to create such diagrams? Would be useful to monitor efforts to reduce the code size such as #960 and #966.

[elsehow]

@webmaster128 I've been using next analyze. I'm not sure what its compatibility is like outside of next.js, but you could always create a minimal next.js app for testing.

[webmaster128]

Very nice, thanks. I can create it locally using this instruction and this next.js project using CosmJS.

[webmaster128]

If you want to get rid of libsodium in your application, you can have a look at this trick from Keplr:

• Resolve to a custom package https://github.com/chainapsis/keplr-wallet/blob/c1976b522f610a9d894de7860dc1889486820bc8/package.json#L81-L82
• Create empty package https://github.com/chainapsis/keplr-wallet/tree/master/etc/noop

This is clearly no ideal for maintenance and it can always happen that your app breaks because CosmJS changes. But right now libsodium is only used for wallet related crypto and Ed25519. So it's an okay trade-off if your app is tested well, production ready and you read CHANGELOGs.

[webmaster128]

Thank you for bringing this up. This ticket triggered many dependency removals in https://github.com/cosmos/cosmjs/milestone/14, the module type discussion in #1004 and the libsodium replacement discussion in #1031. I think with those more specific ticket, we can close here.

Making CosmJS faster and lighter but preserving a wide range of targets will be a continous effort.

Happy for any follow-up input in more specific tickets.