App submission: monero

[deverickapollo]

App Submission

monero

256x256 SVG icon

• https://svgshare.com/i/vGT.svg

Gallery images

• https://raw.githubusercontent.com/deverickapollo/umbrel-monero/master/pictures/Monero%20Dashboard%20Menu.png
• https://raw.githubusercontent.com/deverickapollo/umbrel-monero/master/pictures/Connect%20Wallet.png
• https://raw.githubusercontent.com/deverickapollo/umbrel-monero/master/pictures/Advanced%20Settings.png

I have tested my app on:

• Umbrel dev environment
• Umbrel OS on a Raspberry Pi 4
• Custom Umbrel install on Linux

[m1m3-50]

Please check my comments about versioning for the app itself and also the Monero node image used.

[deverickapollo]

Updates complete. Thanks!

[nmfretz]

Apologies for the delay in reviewing this @deverickapollo! I have had my head down working on other parts of the code-base. I will try and review soon.

Thanks @m1m3-50 and @IMPranshu for stepping up and helping. Much appreciated.

[nmfretz]

Excellent work on this @deverickapollo! This is a fantastic addition for the Monero community. See below for some suggested changes.

[nmfretz]

Is this to override a healthcheck designated in the Dockerfile that occurs super often?

[deverickapollo]

Looking into this now - I believe this is related to some work I'm doing to prepare monerod for an integration with btcpay. I believe we can remove this for now.

[nmfretz]

I installed the app and am syncing the blockchain now. Initial sync is working great right off the bat.

The primary purpose of my review is just to make sure that the app is packaged correctly for Umbrel; however, since I have some familiarity with the UI and server code of your app from working on our Bitcoin Node app, I can offer some additional help.

Check out this video below (no sound) where I highlight a few bugs I found while checking out the app. This is my first time playing around with monero, so apologies if I have misunderstood anything.

monero-bugs.mp4

Notes for the video:

• Outgoing Connections to Tor Peers is enabled in Advanced Settings, but there are no related lines in the bitmonero.conf. You can see in the UI that I have no connections to tor peers, and I confirmed this with RPC /get_connections. You likely need to update your settingsToMultilineConfString() function to write the required tor config to your bitmonero.conf
• The info popup next to the blockchain size is showing up even though Prune Blocks is disabled. This should be an easy conditional fix in the UI.
• Is I2P implemented in your app yet? monerod errors out when enabling Outgoing Connections to I2P Peers. If it's still a work in-progress, it could be worth removing the i2pd container and all associated code from this version so that users don't unwittingly crash their node and need to change both the monero-config.json and bitmonero.conf to get things back up and running.
• When I try to restart the node with Incoming Connections enabled in Advanced Settings, the UI shows an error and doesn't let me do it.
• If I disable Outgoing Connections to Tor Peers and try to restart the node, the UI shows an error. This error handling is included in our Bitcoin Node app to prevent users from disabling connections to all peers on clearnet, tor, and I2P. If they did this, then they would not be able to connect to any peers. However, it looks like there is no toggle to disable connections to clearnet peers in your app, so you just need to update the error logic in the UI to allow both Tor and I2P connections to be disabled at the same time.

This was just from a quick play, so there are probably other sneaky bugs that have understandably resulted from translating the bitcoin app to a monero app.

[nmfretz]

@deverickapollo I noticed that monerod uses quite a lot of RAM. This is during initial block download, with DB Sync Mode set to the fast default. I believe I have seen it go as high as 9GB for me (I'm testing with 16GB RAM). Do you know if monerod frees up RAM for other processes when needed?

CONTAINER ID   NAME                        CPU %     MEM USAGE / LIMIT     MEM %     NET I/O           BLOCK I/O         PIDS
7fab4c46fd05   monero_monerod_1            171.84%   6.07GiB / 15.42GiB    39.37%    25.8GB / 427MB    28.1GB / 294GB    29

I have also noticed that if I try and restart my Umbrel while Monero is installed and running, it can take a very long time to shut down. Then when everything comes back online, the monero app UI can become stuck in this state even though monerod is plugging away in the background:

Monero-restart.mp4

It may be that during late-stage initial block download, monerod can take a while before it accepts RPC calls

[deverickapollo]

@nmfretz - This is great! Thanks for taking the time to review the app! A few of these issues I am aware of. I should be able to address most in the next week. I'll reach out once that work is complete.

[nmfretz]

Excellent, looking forward to it @deverickapollo! I have a monero node happily plugging away on one of my Umbrels, so I can help with any testing you need.

[ceramicwhite]

I'm running this. Fully synced it's sitting at <1gb ram and during syncing I never saw it go above 4gb

[nmfretz]

@deverickapollo let me know whenever you are ready for me to continue reviewing so I don't accidentally hold you up!

[ParthJadhav]

Hey @deverickapollo , just following up on the PR. Is it ready to be reviewed?

[deverickapollo]

@nmfretz - Bumped the app to 1.0.4 which should handle the startup issues noted. Some of the peering nodes defined in the config file were offline and caused the latency during boot. Let me know if you're still experiencing any issues there. If twe're good there, I am ok with shipping the current version.

[nmfretz]

Thanks @deverickapollo, I just tested now and I still have the same issue noted here where monerod isn't accepting rpc calls from the UI during the first X% of IBD.

When I view the monerod logs I see that sync starts right away and progresses normally. It's just that the frontend UI can't get any info back from monerod for quite a while into sync. I tested rpc calls from inside the monerod container and they work fine during initial sync.

What is your preference for launching now with this in the app description:

Please note that after installing Monero Node, the app interface may not display any sync progress for several hours.
This behaviour is typical at this stage of development. The initial sync is still happening in the background,
and you'll see the status reflected in the app as soon as it's available.

versus continuing to troubleshoot?

[deverickapollo]

@nmfretz - That should be fine. I'll investigate those issue and update in a later release.

[tylerwillis]

Any update on when to expect this in the official App Store?

I got this running via community app store, but would love to recommend it to friends without signing up to be their IT Support on how to use community app stores. :)

[nmfretz]

Hey @deverickapollo I thought I had posted this here, but it appears I never did. Sorry about that.

When I was testing last month, I fully synced Monero and noticed that the UI shows 0% synced when monerod is on the latest block. This results in the loading animation permanently running in the Blockchain card and could be confusing to most users. Is this the case for you?

[deverickapollo]

@nmfretz Funny you mention it - I was just playing around with this last night. It turns out once the node is synced, target_height goes to 0 Issue 7029. I was using target_height to track our sync status and some of my recent changes didn't consider this. I should have a fix ready to go soon.

[sethforprivacy]

Please note @deverickapollo that I'll be shutting down my Docker Hub account at some point in the near future, so GHCR should be used for my image here instead, i.e. for v0.18.3.3:

ghcr.io/sethforprivacy/simple-monerod:v0.18.3.3

[deverickapollo]

Appreciate the heads up! I'll get that change in asap

[nmfretz]

Thanks a bunch for noting that @sethforprivacy 🤝

[deverickapollo]

Monerod version change complete. Fix for sync % displayed on startup and full sync also added

[markisoke]

@deverickapollo Any way you will have the option to enable ZMQ in your app for us P2Pool users?

[nmfretz]

Thanks @deverickapollo. I still get the same original issue where sync percent doesn't show in the UI:

Can you confirm whether you have this same issue?

[highghlow]

+1 for this. Monero is the best crypto and a dedicated server is the best way to sync it

[highghlow]

Thanks @ deverickapollo. I still get the same original issue where sync percent doesn't show in the UI:
Can you confirm whether you have this same issue?

I've tested the app myself and can confirm that the issue is present. While the node is syncing, no progress gets displayed

[deverickapollo]

@highghlow - thanks for confirming.

I'll review this asap. I just completed a refactor on the backend to replace monero-javascript with monero-ts so I'll include those changes and retest.

[nmfretz]

Thanks @deverickapollo, ping me when you're ready!

[highghlow]

Update: it updates the sync progress when Umbrel is rebooted

[nmfretz]

Hey @deverickapollo, out of curiosity I did a deep dive into this issue this morning. Hopefully this will help you with a fix!

TLDR:

1. Keep at least --rpc-bind-ip=0.0.0.0 in the monerod start command so that the external server container can make RPC calls even before your custom bitmonero.conf is sourced.
2. Make sure that the server can restart monerod on initial boot so that the bitmonero.conf is sourced on app install. Right now, it fails to do so even with Fix (1) noted above.

After that this is good to go from a packaging standpoint!

1) First, the general RPC issue where the UI cannot make any calls to monerod:

In version 1.0.1 you removed these cli arguments that used to be passed to the monerod command on container start, and placed their equivalents in the logic that creates the default bitmonero.conf:

From exports.sh: deverickapollo@aa48aaa

BIN_ARGS=()
# Commenting out options that are replaced by generated config file. We should migrate all these over in a future update.
# BIN_ARGS+=( "--rpc-bind-port=${APP_MONERO_RPC_PORT}" )
# BIN_ARGS+=( "--rpc-bind-ip=0.0.0.0" )
# BIN_ARGS+=( "--confirm-external-bind" )

From umbrel-monero logic/disk.js: deverickapollo/umbrel-monero@2c612fb

  umbrelMoneroConfig.push(`rpc-bind-ip=0.0.0.0`);
  umbrelMoneroConfig.push(`rpc-restricted-bind-ip=0.0.0.0`);
  umbrelMoneroConfig.push(`rpc-bind-port=${constants.MONERO_RPC_PORT}`);
  umbrelMoneroConfig.push(`rpc-restricted-bind-port=${constants.MONERO_RESTRICTED_RPC_PORT}`);

This change causes issues because there isn't actually any bitmonero.conf when the app is first installed. It's created by the server container after initial boot, so without a custom bitmonero.conf or rpc-bind-ip cli argument, monerod actually defaults to binding 127.0.0.1 instead of 0.0.0.0. This means that applications outside of the monerod container (like the server container) can't make RPC calls. You won't see this issue when testing locally on your computer because the docker-compose.yml file in the umbrel-monero repo includes --rpc-bind-ip=0.0.0.0 in the monerod container command.

There is logic in the server container to restart monerod when the bitmonero.conf file is created (see next point below for a second issue), but without rpc-bind-ip=0.0.0.0 on initial boot of monerod, the server can't actually send an RPC call to tell monerod to restart.

@highghlow is right that when you restart umbrel, the UI will then start showing sync progress. And actually it's just that when you restart the monerod container, progress will be shown, because monerod will then source the bitmonero.conf that the server container created. You can test this by installing monero and seeing that no RPC calls can be made externally. Then run:
sudo docker restart monero_monerod_1
to restart the container. monerod will then source the bitmonero.conf and the UI will show progress. You can even try making manual RPC calls to the monerod container.

2) Even when starting monerod with rpc-bind-ip=0.0.0.0, the server container doesn't seem to be able to restart monerod when it is first booting up.

Once monerod has initialized and the UI shows data, you can restart monerod from Advanced Settings. However, when the app is first installed you will see this error:

> umbrel-middleware@0.4.0 start
> node ./bin/www

Listening on port 9976
bitmonero.conf does not exist, creating config files with default values
MoneroError: RequestError: Error: connect ECONNREFUSED 10.21.21.179:18081
    at MoneroRpcConnection.sendPathRequest (/app/node_modules/monero-javascript/src/main/js/common/MoneroRpcConnection.js:320:18)
    at async MoneroDaemonRpc.stop (/app/node_modules/monero-javascript/src/main/js/daemon/MoneroDaemonRpc.js:785:16)
    at async onmessage (/app/node_modules/monero-javascript/src/main/js/common/MoneroWebWorker.js:44:49) {
  code: undefined
}

The server will keep trying to restart monerod for 1 minute in order to source the newly created bitmonero.conf:

bin/www:

async function createConfFilesAndRestartMonerod() {
  console.log('bitmonero.conf does not exist, creating config files with default values');
  const config = await diskLogic.getJsonStore();

  await diskLogic.applyCustomMoneroConfig(config, true);

  const MAX_TRIES = 60;
  let tries = 0;

  while (tries < MAX_TRIES) {
    try {
      await monerodLogic.stop();
      break;
    } catch (error) {
      console.error(error);
      tries++;
      await new Promise((resolve) => setTimeout(resolve, 1000));
    }
  }
}

I haven't explored this one further, but it is possible that monerod just isn't ready to accept RPC requests this early on in the initial boot. Without this initial restart, users will be running monerod with default config values and not inheriting anything you have set as your initial bitmonero.conf. They would only source the bitmonero.conf if they restart the app, restart umbrel, or change settings and restart through Advanced Settings within the monero app.

[nmfretz]

Hey @deverickapollo ping me when you are ready for me to test!

[deverickapollo]

@nmfretz Appreciate that additional input! I was able to reproduce this issue and identify the problem. I've included a fix in the latest release that should address the startup issue.

[nmfretz]

Well done @deverickapollo! This is good to go from a packaging perspective. Welcome to the official app store 🎉