Migrating/Integrating WebCrypto (possibly WASM) instead of Node Forge Crypto

[CMCDragonkai]

Specification

The work in #155 involving nativescript meant that we are starting to consider alternatives to Node forge crypto in order to standardise our crypto API and future proof to future JS-based platforms. There are number of reasons to look into webcrypto:

• WebCrypto is a standard across ES platforms, thus enabling more potential native-implementations which can improve performance and security (prevent timing attacks)
• WebCrypto API focuses on using ES-compliant buffers like Uint8Array and ArrayBuffer which can help us standardise our buffer usage across js-db, js-id, js-workers and more, especially as workers require ArrayBuffer to do zero-copy
• There is existing work with WebCrypto API involving WASM to plug in functionality that isn't supported by the standard, such as providing ed25519 keys, and this will help us resolve the usage of ed25519 Replace node-forge RSA Keypair With ed25519/x25519 Keypair #168
• The node-forge source code isn't well maintained and doesn't have as many eyes watching and evaluating its security

Now for some background:

This comment #43 (comment) explains how we came to be using node-forge as opposed to other cryptographic libraries.

I know that when Polykey project first started, we initially were thinking of using PGP and thus the kbpgp.js library (https://github.com/keybase/kbpgp). We ended up going away from PGP due to its limited usage in a number of scenarios that we want PK to deal with. Namely end to end encrypted network communication which is a TLS issue, that makes use of X.509 certificates rather than PGP certificates. Furthermore we also had symmetric encryption/decryption scenarios like js-encryptedfs that again would not make use of PGP standards. Therefore it just lacked interoperability with many other cryptographic scenarios, it seemed like its own island of standards, the library can still be brought back in in the future if we find usecases for PK using PGP.

However in choosing node-forge, we came across a few other problems. Mainly overall-cross platform compatibility planning for mobile devices. This is not just a problem with crypto, but also other libraries that are used in our networking domain such as utp-native.

Here are something I found that may be relevant to us proceeding here:

• The webcrypto standard (https://w3c.github.io/webcrypto/) is now the official standard for cryptographic operations in browser environments. It is now being supported by all major browsers and electron
• This webcrypto standard is now taking over other deployment platforms such as in Nodejs: https://nodejs.org/api/webcrypto.html & https://www.nearform.com/blog/implementing-the-web-cryptography-api-for-node-js-core/
• As a standardised API, switching over to this API gives us more cross-platform opportunity as the rest of the world's development ecosystem catches up and migrates over to webcrypto standard.
• In particular are nativescript and react-native ecosystems. Currently none of them have official crypto APIs, they expect the developer to use underlying native iOS or Android crypto APIs.
• However I found out that the GUN project https://github.com/amark/gun#additional-cryptography-libraries claims to be able to use webcrypto on react-native and other platforms.
• It does this through a "webview" trick. Basically this hooks into the browser runtimes that are on iOS and Android to perform the actual crypto and then return the results back to the main application. I have no idea about the performance characteristics of this trick. This trick is described here: https://gun.eco/docs/React-Native and https://github.com/webview-crypto/react-native-webview-crypto
• The usage of webcrypto libraries is pioneered by "PeculiarVentures" which has these main libraries:
  • https://github.com/PeculiarVentures/PKI.js - this could be an alternative to node-forge for all of our PKI/X.509 and TLS related functionality (previously it was claimed this was overly complex compared to node-forge)
  • https://github.com/PeculiarVentures/webcrypto - this is a generic polyfill for webcrypto, not entirely sure how it wraps the native webcrypto inside nodejs or if smooths over the differences, there is a discussion about this library in relation to other webcrypto polyfills targeting nodejs https://github.com/PeculiarVentures/webcrypto, because this is also mentioned with respect to the webview trick, there may also be a relationship between the webview trick and this polyfill
  • They have many other crypto related libraries that we should investigate

All of this will mean that we either replace node-forge, or end up creating a adapter pattern where we plugin different crypto implementions depending on our environment. At this point in time, the keys domain abstracts over most(all?) crypto operations for all other domains. Except in the case of EFS which is currently pinned to node-forge (it may be a good idea to abstract that and expect an interface of functions for EFS).

Cross platform compatibility here isn't just about the fundamental crypto library. It's also about other parts of PK. One closely related situation is the JOSE libraries. As they involve cryptographic operations, they currently seem to "fix" their underlying crypto library as well. It would be ideal that if we standardise on a crypto library for cross-platform deployment, that we can also ensure that our JOSE library is using the same crypto library to reduce our crypto attack surface. We are currently using https://github.com/panva/jose which uses native crypto depending on the platform including webcrypto. Contenders include https://github.com/cisco/node-jose (which fixes on node-forge) and https://github.com/square/js-jose.

Additional context

• Make this workable ES-compliant platforms (like browsers) by generalising crypto and performance APIs js-id#2 - js-id also should be using WebCrypto
• React Native and Mobile OS (iOS and Android) Compatibility #155 - will impact NativeScript deployment
• Use bufferWrap to support ArrayBuffer, Uint8Array and Buffer js-db#3 - can help make use of the migration to ArrayBuffer and typed arrays
• WebAssembly resources
  • There are 2 well supported runtimes for WASM:
    • wasm3 - supports interpretation
    • wasmer - supports JIT and AOT compilation
  • https://www.fastly.com/blog/standard-cryptography-api-webassembly
  • https://github.com/WebAssembly/wasi-crypto
  • https://stackoverflow.com/questions/48891593/webcryptoapi-vs-webassembly-encryption-module
  • https://medium.com/nexenio/a-random-story-on-webcrypto-and-webassembly-6f9e00c73a - This describes how to use webcrypto together with webassembly when you need a custom crytographic functionality not supported inside webcrypto.
  • https://medium.com/cubbit/how-to-build-a-crypto-isomorphic-library-with-javascript-and-webassembly-6fc7aa708437
  • https://github.com/cubbit/enigma and WASM in Enigma cubbit/enigma#16

Wasmer can compile WASM code to native code. But wasm3 is for interpretation. Why use interpetation?

It appears that in some cases interpretation can be more widely deployed. There are examples of iOS apps using wasm3. https://github.com/kateinoigakukun/wasmic-ios

It's becoming fast a standard target for many languages. Even TypeScript when ported to AssemblyScript can be compiled to WASM.

Once it is WASM, the only thing missing is broad adoption of WASI. If WASI is broadly adopted like it is in nodejs (https://github.com/nodejs/uvwasi), then pretty much we have a universal portable binary capable of doing relevant system operations. WASI is like a universal standard of system calls. Like a whole new POSIX standard.

Then one would just use WASM and WASI for all platforms.

Tasks

1. ...
2. ...
3. ...

[joshuakarp]

Note that we currently depend on crypto within the sigchain with our usage of the JWS in panva-jose. We use it to both sign and verify a claim (requiring asymmetric cryptography, which in turn requires generating a KeyLike from the PEM-formatted public and private key). This would need to be refactored in order to migrate away from crypto.

Original discussion of this can be found here https://gitlab.com/MatrixAI/Engineering/Polykey/js-polykey/-/merge_requests/213#note_736387549

[CMCDragonkai]

Note that age and wireguard both use https://en.m.wikipedia.org/wiki/ChaCha20-Poly1305.

It's an alternative to AES GCM which has higher performance when the hardware doesn't support AES-NI instructions.

AES GCM is still good for compliance though, some places like government require this.

But the algorithm apparently works and is designed for software-only implementations.

[CMCDragonkai]

We will need to elevate this issue to a higher priority, we now have empirical results on how long it takes to generate a root key in multiple arenas:

1. Tests
2. AWS fargate

On fargate, using the 0.25 CPU containers, it takes 20 minutes to generate a root key pair. Se can be seen by this cloudwatch cpu utilisation graph:

At the same time, key generation is using all cores by default. This doesn't even require the worker manager to be integrated, I think node-forge does this by default.

We believe that this is causing CPU starvation of all the other testing workers that jest creates and thus leading to test timeouts as can be seen in #394.

As we have deployed our testnet, and we expect the need to autoscale the agents, key generation is going to be an important workload, as it delay the scaling up process. Remember even if we pass in a recovery code, the keys must still be generated and this still takes time.

Furthermore we want to get #168 done before going to mainnet, because at that point users will be using our system, and we don't want to get stuck on legacy RSA.

[CMCDragonkai]

@tegefaulkes @emmacasolin

[tegefaulkes]

It's possible to override the number of workers that the generateKeyPair src/keys/utils.ts:64 uses. We can provide workers: 1 as part of the options.

[CMCDragonkai]

The node-forge has its own workers implementation it doesn't actually use WorkerManager. We are looking into refactoring WorkerManager with respect to the Queue and Scheduler implementation in #329. And replacing the crypto is important for our (second, our first one was back in around Jan - Mar 2021 which failed) beta launch, because we want users to be using #168.

So this is good opportunity to take advantage of some intersectionality between crypto, workers, and task abstraction.

[CMCDragonkai]

Note that 16.17 has proper support for ed25519:

Current latest 22.05 is still 16.16.

Will require going to unstable if we want 16.17's version of ed25519.

[CMCDragonkai]

The commit that introduced 16.17 into nodejs is 6e2536f1b09863e984f0479ea4b162e4fe86493d.

To use it, one must use nodejs-16_x as nodejs will be 18.9.0 which may have some major differences.

I'm going to try the latest master commit and see how we go. This will impact all of the nix-derived dependencies.

[CMCDragonkai]

There is 7 points remaining for this epic (since they had work done already).

• Replace node-forge RSA Keypair With ed25519/x25519 Keypair #168 - 2 points
• Password decrypt the --root-key-file #433 - 1 point
• Slice out KeyManager into 2 parts: KeyRing, CertificateManager #472 - 4 points

[CMCDragonkai]

The benchmark results in #446 means that we do need webcrypto and due to iOS issues, we cannot use wasm. So we have moved to libsodium instead, and the benchmarks show that libsodium is 10x to 50x faster than webcrypto.

We are also going with native Buffer, and future usage will just use the Buffer polyfill.

There remains 2 uses of webcrypto:

1. bip39 recovery code
2. x509 signing

The first can be replaced with our own KDF mechanism. In that sense, we would no longer use bip39.

The second is a bit more difficult, it may be easier to create a webcrypto shim backed off libsodium and only for ed25519 signing.