[feature] Make it easier to use monero-javascript for front end development #83

CryptoGrampy · 2022-03-22T16:18:44Z

For such a comprehensive and easy to use library for node, it's not possible to simply npm i monero-javascript and use it for front end/browser apps (yes, this is documented), and it's quite painful to integrate it with even basic create-react-app or vuejs boilerplate code. I think this is a big problem and is likely a bottleneck for a lot of would-be front end development.

I would really like to be able to simply install and use this library with no fussing with webpack and build configs if such a thing is possible.... i.e. I should be able to run create-react-app, or npm init vue@latest, run npm i monero-javascript, and be ready to roll.

Example of what needs to be done to use the library with Vue (see the fallbacks and additional packages necessary in package.json): https://github.com/xmrdotgift/wallet/blob/master/webpack.config.js

woodser · 2022-03-22T18:00:45Z

I support this and would also like to see typescript implemented.

I'm looking for help on both. Thanks.

trasherdk · 2022-03-23T23:19:50Z

I was looking to use this thing in sveltekit, but that seems to be blocked by the vite tooling.

vitejs/vite#5075 links to a bunch of issues describing the problem.

It's not really a monero-javascript problem, as it's more a tooling not being ready for wasm problem, I think.

Edit: Maybe this Using WASM in SvelteKit can bring me closer.

j-berman · 2022-03-24T23:36:54Z

I've been messing around with this off and on. Here's a workaround to get something going with create-react-app:

npx create-react-app <app name>
cd <app name>

# necessary to avoid needing to eject or override create-react-app's defaults
# see: https://github.com/facebook/create-react-app/issues/11756
npm uninstall react-scripts && npm install react-scripts@4

npm i monero-javascript

cp node_modules/monero-javascript/dist/* public
cp node_modules/monero-javascript/src/main/js/common/MoneroWebWorker.js public/MoneroWebWorker.js

trasherdk · 2022-04-04T16:50:40Z

I'm hoping svelte-emscripten is up for the task.

The required emscripten linker flags got me worried, as I can't find those. What do you think @woodser ?

✔️ -s MODULARIZE=1
❌ -s ENVIRONMENT='web'
❌ -s EXTRA_EXPORTED_RUNTIME_METHODS="['specialHTMLTargets', 'JSEvents', 'GL', 'callMain', 'abort']"

How to Use

When you compile your program in Emscripten, use the output filetype *.js and use the linker flags below. These are required to adapt your program into component form:

    -s MODULARIZE=1

    -s ENVIRONMENT='web'

    -s EXTRA_EXPORTED_RUNTIME_METHODS="['specialHTMLTargets',
           'JSEvents', 'GL', 'callMain', 'abort']"

Then, use the component in your Svelte app:

import { default as Emscripten } from 'svelte-emscripten';
import { default as module } from './module.js';

<Emscripten
  module={module}
  canvas={true}
  console={false}
  verticalOrientation={false}
  options={
    { 
      autorun: true,
      wasmPath: 'path/to/module.wasm'
    }
  }
/>

woodser · 2022-04-08T12:46:16Z

The required emscripten linker flags got me worried, as I can't find those.

These linker flags are defined here.

You get an error if you add them? What's the error?

Sometimes external nodejs dependencies need to be excluded from the build, here.

trasherdk · 2022-04-09T09:13:32Z

I'm not getting far enough to start tweaking linker flags.

Trying to build standard on Slackware 14.2 (Intel x64) ends with:

-- Configuring done
-- Generating done
-- Build files have been written to: /path/to/build-monero-javascript/github-monero-javascript/build
make: cmake --build . -j4
Unknown argument -j4
Usage: cmake --build <dir> [options] [-- [native-options]]
Options:
  <dir>          = Project binary directory to be built.
  --target <tgt> = Build <tgt> instead of default targets.
  --config <cfg> = For multi-configuration tools, choose <cfg>.
  --clean-first  = Build target 'clean' first, then build.
                   (To clean only, use --target 'clean'.)
  --use-stderr   = Ignored.  Behavior is default in CMake >= 3.0.
  --             = Pass remaining options to the native tool.
emmake: error: 'cmake --build . -j4' failed (returned 1)

trasherdk · 2022-04-09T09:18:07Z

The build script:

#!/bin/bash

BASE=$(dirname "$(realpath "$0")")

cd "${BASE}" || exit 1

if [ ! -d "${BASE}/emsdk" ]; then
    git clone https://github.com/emscripten-core/emsdk.git
fi

cd "${BASE}/emsdk" || exit 1

git pull \
&& ./emsdk install latest-upstream \
&& ./emsdk activate latest-upstream \
&& source ./emsdk_env.sh

export EMSCRIPTEN="${BASE}/emsdk/upstream/emscripten"
export EMSDK="${BASE}/emsdk"
export EM_CONFIG="${BASE}/emsdk/.emscripten"
export EMSDK_NODE="${BASE}/emsdk/node/14.18.2_64bit/bin/node"

cd "${BASE}/github-monero-javascript" || exit 1

./bin/update_submodules.sh || exit 1

PATCH_FILE="${BASE}/github-monero-javascript/external/monero-cpp/external/monero-project/src/crypto/wallet/CMakeLists.txt"
sed -i 's/set(MONERO_WALLET_CRYPTO_LIBRARY "auto"/set(MONERO_WALLET_CRYPTO_LIBRARY "cn"/g' "${PATCH_FILE}"

HEADER_FILE="${BASE}/../build-monero-pool/staging-area/monero/monero-build/translations/translation_files.h"
HEADER_DEST="${BASE}/github-monero-javascript/external/monero-cpp/external/monero-project/build/release/translations/"

if [ ! -f "${HEADER_DEST}/$(basename "${HEADER_FILE}")" ]; then
    echo "*${WHITE} copy 'translation_files.h' ${RESTORE}"
    sleep 3
    cp "${HEADER_FILE}" "${HEADER_DEST}"
fi

./bin/build_all.sh || exit 1

Edit 1: Forgot to patch MONERO_WALLET_CRYPTO_LIBRARY
Edit 2: Copy translation_files.h into include path

trasherdk · 2022-04-09T11:47:28Z

Removing -j$HOST_NCORES from bin/build_wasm_emscripten.sh moves past that error.

Next error is:

monero-project/src/common/i18n.cpp:36:10: fatal error: 'translation_files.h' file not found
#include "translation_files.h"

That file does not exist in external/monero-cpp/external/monero-project/translations/

Isn't it supposed to be generated by:
external/monero-cpp/external/monero-project/translations/generate_translations_header.c ?

woodser · 2022-04-09T12:05:38Z

That file does not exist in external/monero-cpp/external/monero-project/translations/

I'd try building the monero-project submodule directly to ensure it succeeds, and build twice since some files are not built the first time, e.g.:

cd ./external/monero-cpp/external/monero-project
make release-static -j4
make release-static -j4

Then the translations directory should exist.

trasherdk · 2022-04-09T12:54:20Z

Building monero in ./bin/build_all.sh should be after boost has been build.
Otherwise it will fail if system boost does not have static libraries installed.

Maybe setting BOOST_ROOT to build/boost will do the trick.

trasherdk · 2022-04-09T13:38:39Z

Nope, no cake.

BOOST_ROOT=$(realpath "/path/to/github-monero-javascript/build/boost") \
BOOST_LIBRARYDIR="${BOOST_ROOT}/lib/" \
make release-static -j4

results in:

CMake Error at /usr/share/cmake-3.5/Modules/FindBoost.cmake:1657 (message):
  Unable to find the requested Boost libraries.

  Boost version: 1.59.0

  Boost include path: /usr/include

trasherdk · 2022-04-09T14:34:53Z

Building monero standalone is just fine.

[ 98%] Built target gen_multisig
[ 98%] Building CXX object src/daemon/CMakeFiles/daemon.dir/command_server.cpp.o
[ 98%] Building CXX object src/daemon/CMakeFiles/daemon.dir/daemon.cpp.o
[100%] Building CXX object src/daemon/CMakeFiles/daemon.dir/executor.cpp.o
[100%] Building CXX object src/daemon/CMakeFiles/daemon.dir/main.cpp.o
[100%] Building CXX object src/daemon/CMakeFiles/daemon.dir/rpc_command_executor.cpp.o
[100%] Linking CXX executable ../../bin/monerod
[100%] Built target daemon
make :  OK

Building in external is not. Tried woodser/monero and that failed too.

CMake Error at /usr/share/cmake-3.5/Modules/FindBoost.cmake:1657 (message):
  Unable to find the requested Boost libraries.

  Boost version: 1.76.0

  Boost include path:
  /home/crypto/local/build/build-monero-javascript/github-monero-javascript/build/boost/include


  Could not find the following Boost libraries:

          boost_filesystem
          boost_date_time
          boost_program_options
          boost_locale

Changing bin/build_boost_emscripten.sh

./bootstrap.sh \
- --with-libraries=system,thread,chrono,serialization,regex \
+ --with-libraries=system,thread,chrono,serialization,regex,filesystem,date_time,program_options,locale \
2>&1

Results in:

CMake Error at /usr/share/cmake-3.5/Modules/FindBoost.cmake:1657 (message):
  Unable to find the requested Boost libraries.

  Boost version: 1.76.0

  Boost include path:
  /home/crypto/local/build/build-monero-javascript/github-monero-javascript/build/boost/include


  Could not find the following Boost libraries:

          boost_locale

trasherdk · 2022-04-09T14:36:45Z

The updated build script:

#!/bin/bash

BASE=$(dirname "$(realpath "$0")")

cd "${BASE}" || exit 1

if [ ! -d "${BASE}/emsdk" ]; then
    git clone https://github.com/emscripten-core/emsdk.git
fi

cd "${BASE}/emsdk" || exit 1

git pull \
&& ./emsdk install latest-upstream \
&& ./emsdk activate latest-upstream \
&& source ./emsdk_env.sh

export EMSCRIPTEN="${BASE}/emsdk/upstream/emscripten"
export EMSDK="${BASE}/emsdk"
export EM_CONFIG="${BASE}/emsdk/.emscripten"
export EMSDK_NODE="${BASE}/emsdk/node/14.18.2_64bit/bin/node"

cd "${BASE}/github-monero-javascript" || exit 1

./bin/update_submodules.sh || exit 1

echo "${WHITE}* Patching ${YELLOW}build_boost_emscripten.sh${RESTORE}"

PATCH_FILE="${BASE}/monero-javascript/bin/build_boost_emscripten.sh"
PATCH_FROM='--with-libraries=system,thread,chrono,serialization,regex \\'
PATCH_TO='--with-libraries=system,thread,chrono,serialization,regex,locale,filesystem,date_time,program_options \\'
sed -i "s/${PATCH_FROM}/${PATCH_TO}/g" "${PATCH_FILE}" || exit 1

./bin/build_boost_emscripten.sh || exit 1

PATCH_FILE="${BASE}/github-monero-javascript/external/monero-cpp/external/monero-project/src/crypto/wallet/CMakeLists.txt"
sed -i 's/set(MONERO_WALLET_CRYPTO_LIBRARY "auto"/set(MONERO_WALLET_CRYPTO_LIBRARY "cn"/g' "${PATCH_FILE}"

MONERO_BUILD="${BASE}/github-monero-javascript/external/monero-cpp/external/monero-project/build"
if [ ! -d "${MONERO_BUILD}" ]; then
    mkdir "${MONERO_BUILD}" || exit 1
fi

cd "${MONERO_BUILD}" || exit 1

export BOOST_ROOT="${BASE}/github-monero-javascript/build/boost"
export BOOST_LIBRARYDIR="${BOOST_ROOT}/lib"
cmake .. || exit 1
make release-static -j2 || exit 1
make release-static -j2 || exit 1

#HEADER_FILE="${BASE}/../build-monero-pool/staging-area/monero/monero-build/translations/translation_files.h"
#HEADER_DEST="${BASE}/github-monero-javascript/external/monero-cpp/external/monero-project/build/release/translations/"

#if [ ! -f "${HEADER_DEST}/$(basename "${HEADER_FILE}")" ]; then
#    echo "*${WHITE} copy 'translation_files.h' ${RESTORE}"
#    sleep 3
#    cp "${HEADER_FILE}" "${HEADER_DEST}"
#fi

cd "${BASE}/github-monero-javascript/" || exit 1

./bin/build_all.sh || exit 1

Edit 1: submodules before boost :)
Edit 2: patch build_boost_emscripten.sh
Edit 3: Fix patch build_boost_emscripten.sh :)

trasherdk · 2022-04-09T17:09:27Z

Okay, this is beyond me. I can't get that locale thing to build.

woodser · 2022-04-09T20:20:10Z

It shouldn't be necessary to modify the boost libraries in bin/build_boost_emscripten.sh.

You still get boost errors without that modification? And have you installed boost with sudo apt install libboost-all-dev?

trasherdk · 2022-04-10T03:58:44Z

I'm on Slackware 14.2 (x64). Slackware does not include static libraries at all.

DangerOnTheRanger · 2022-04-11T11:30:50Z

What would need to change in order to remove the need to copy files out of dist/ when targeting the browser? I'm interested in fixing this.

CryptoGrampy · 2022-04-12T03:58:28Z

What would need to change in order to remove the need to copy files out of dist/ when targeting the browser? I'm interested in fixing this.

This is a bit out of my wheelhouse, but I did a little searching on npm and found a couple examples where they're providing their wasm as browser-importable js modules:
https://www.npmjs.com/package/brotli-wasm
https://github.com/jungomi/xxhash-wasm

Would be great if there was just one minified / tree-shakable js file that just works on browser or node. We're definitely almost there. Searching for libraries on npm with 'browser and node.js' might give some good examples of what might need to be done. Axios is one that's usable on node and browser without any user-involvement beside installing, and it just works.

DangerOnTheRanger · 2022-04-12T10:37:41Z

What would need to change in order to remove the need to copy files out of dist/ when targeting the browser? I'm interested in fixing this.

This is a bit out of my wheelhouse, but I did a little searching on npm and found a couple examples where they're providing their wasm as browser-importable js modules: https://www.npmjs.com/package/brotli-wasm https://github.com/jungomi/xxhash-wasm

Would be great if there was just one minified / tree-shakable js file that just works on browser or node. We're definitely almost there. Searching for libraries on npm with 'browser and node.js' might give some good examples of what might need to be done. Axios is one that's usable on node and browser without any user-involvement beside installing, and it just works.

I think you're right, the solution involves bundling plus a package.json that points to the right files (xxhash for example). I think automating the emscripten part of the process to the point where all one needs to do after a fresh git clone should be to run npm run build-wasm should also be a goal, but maybe others have different thoughts on that. Assuming we do that too, the solution then looks like:

Automate building wasm bindings (this can be done with Docker and land as a separate PR if need be)
Bundle/tree-shake built JS with something like rollup
Point to built CJS/UMD/ESM modules with the package.json that gets published to npm

Having laid out everything like this I don't think it's a ton of work (automating all the wasm stuff would probably be the hardest part), but it's possible I've missed something.

CryptoGrampy · 2023-03-27T15:46:34Z

@woodser @DangerOnTheRanger
Apparently, it's possible to generate WASM as a pre-bundled JS file rather than having separate .WASM that needs to be manually copied by downstream library consumers. This would be a huge benefit for monero-javascript browser library users as they would simply import monero-javascript like any other JS library. MyMonero had a very similar setup with their libraries, and they migrated to a 'unified' WASM/JS file in this commit: mymonero/mymonero-utils@07d6a94

woodser · 2023-03-31T17:21:18Z

@CryptoGrampy I made the change from the PR and created single .js files. However, the .js file for the full wallet is 11.2 mbs, whereas the combined size of the old .js and .wasm is 6.7 mbs. Additionally, monero_web_worker.js punches out to 11.6 mbs, instead of the old 2.2 mbs. I need to find a way to optimize these.

woodser · 2023-10-03T17:36:23Z

Hopefully this issue is resolved with the latest v0.9.2 release (and project rename to monero-ts).

woodser · 2024-09-18T16:14:58Z

A lot of progress has been made since this issue was opened.

Yes, polyfills are still required to use functionality from Node.js in the browser, and I don't see how that can be easily changed. However, numerous other fixes have been made to make this library more friendly with various app frameworks.

First, as requested, we've updated the deprecated networking libraries to Axios, thanks to @mainnet-pat in #230.

Second, as requested, we've combined the dist assets into singular *.js files, completely removing the separate .wasm files, again thanks to @mainnet-pat in #250. Additionally, the dist assets no longer need to be copied manually to "public" (though the web worker must be copied or imported with a special line).

Third, we now support a variety of browser frameworks with sample applications to get up and running quickly, with the help of @mainnet-pat:

What's not supported from the wishlist:

Removing polyfills, but the sample apps clearly demonstrate how to use them.
I haven't tested on Slackware :P

But with the networking libraries updated, the dist assets combined, and the sample apps supported, I'm going to go ahead and close this issue. Please if you encounter any sticking points, open a new issue for us to work through that specifically.

Thanks, and hope to see you guys around! :)

woodser added the help wanted Extra attention is needed label Mar 22, 2022

This was referenced Sep 1, 2024

Combine wasm + js files #247

Closed

Combine .wasm and .js files [3 XMR] #248

Closed

woodser closed this as completed Sep 18, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[feature] Make it easier to use monero-javascript for front end development #83

[feature] Make it easier to use monero-javascript for front end development #83

CryptoGrampy commented Mar 22, 2022 •

edited

Loading

woodser commented Mar 22, 2022

trasherdk commented Mar 23, 2022 •

edited

Loading

j-berman commented Mar 24, 2022

trasherdk commented Apr 4, 2022

woodser commented Apr 8, 2022

trasherdk commented Apr 9, 2022

trasherdk commented Apr 9, 2022 •

edited

Loading

trasherdk commented Apr 9, 2022

woodser commented Apr 9, 2022

trasherdk commented Apr 9, 2022

trasherdk commented Apr 9, 2022

trasherdk commented Apr 9, 2022

trasherdk commented Apr 9, 2022 •

edited

Loading

trasherdk commented Apr 9, 2022

woodser commented Apr 9, 2022

trasherdk commented Apr 10, 2022

DangerOnTheRanger commented Apr 11, 2022

CryptoGrampy commented Apr 12, 2022

DangerOnTheRanger commented Apr 12, 2022

CryptoGrampy commented Mar 27, 2023

woodser commented Mar 31, 2023

woodser commented Oct 3, 2023

woodser commented Sep 18, 2024

[feature] Make it easier to use monero-javascript for front end development #83

[feature] Make it easier to use monero-javascript for front end development #83

Comments

CryptoGrampy commented Mar 22, 2022 • edited Loading

woodser commented Mar 22, 2022

trasherdk commented Mar 23, 2022 • edited Loading

j-berman commented Mar 24, 2022

trasherdk commented Apr 4, 2022

How to Use

woodser commented Apr 8, 2022

trasherdk commented Apr 9, 2022

trasherdk commented Apr 9, 2022 • edited Loading

trasherdk commented Apr 9, 2022

woodser commented Apr 9, 2022

trasherdk commented Apr 9, 2022

trasherdk commented Apr 9, 2022

trasherdk commented Apr 9, 2022

trasherdk commented Apr 9, 2022 • edited Loading

trasherdk commented Apr 9, 2022

woodser commented Apr 9, 2022

trasherdk commented Apr 10, 2022

DangerOnTheRanger commented Apr 11, 2022

CryptoGrampy commented Apr 12, 2022

DangerOnTheRanger commented Apr 12, 2022

CryptoGrampy commented Mar 27, 2023

woodser commented Mar 31, 2023

woodser commented Oct 3, 2023

woodser commented Sep 18, 2024

CryptoGrampy commented Mar 22, 2022 •

edited

Loading

trasherdk commented Mar 23, 2022 •

edited

Loading

trasherdk commented Apr 9, 2022 •

edited

Loading

trasherdk commented Apr 9, 2022 •

edited

Loading