core_doc_update (#851)

ton-community · Nov 13, 2024 · 4c489f7 · 4c489f7
1 parent 2ca4954
commit 4c489f7
Show file tree

Hide file tree

Showing 7 changed files with 224 additions and 1 deletion.
diff --git a/cspell.json b/cspell.json
@@ -157,6 +157,7 @@
     "deeplink",
     "dnsresolve",
     "dracones",
+    "dstat",
     "ed25519_chksign",
     "exitcode",
     "extracurrencies",
@@ -180,11 +181,14 @@
     "hmac",
     "howto",
     "HOWTO",
+    "htop",
     "idict",
+    "iftop",
     "inclusivity",
     "inplace",
     "int",
     "ints",
+    "iotop",
     "ipfs",
     "janvier",
     "jetton",
@@ -221,6 +225,7 @@
     "nanotons",
     "newkeypair",
     "nextra",
+    "nmon",
     "nonexist",
     "nonfinal",
     "penalising",

diff --git a/docs/v3/documentation/infra/nodes/node-commands.mdx b/docs/v3/documentation/infra/nodes/node-commands.mdx
@@ -43,6 +43,7 @@ This document describes the various flags and options available when running a T
 | `--celldb-cache-size`                | Sets the block cache size for RocksDb in CellDb, in bytes.                                                       | `1G` (1 Gigabyte)             | `--celldb-cache-size <size>`                    |
 | `--celldb-direct-io`                 | Enables direct I/O mode for RocksDb in CellDb (only applies when cache size is >= 30G).                          | Disabled                      | `--celldb-direct-io`                            |
 | `--celldb-preload-all`               | Preloads all cells from CellDb on startup.                                                                       | Disabled                      | `--celldb-preload-all`                          |
+| `--celldb-in-memory`                 | Stores the whole celldb in memory. For validators with default settings, size of celldb ~80-100GB, so 128 Gb is absolutely necessary and 256 is preferred.| Disabled|  `--celldb-in-memory`   |
 | `--catchain-max-block-delay`         | Sets the delay before creating a new catchain block, in seconds.                                                 | `0.4` seconds                 | `--catchain-max-block-delay <seconds>`          |
 | `--catchain-max-block-delay-slow`    | Sets the maximum extended catchain block delay for too long rounds, in seconds.                                  | `1.0` seconds                 | `--catchain-max-block-delay-slow <seconds>`     |
 | `--fast-state-serializer`            | Enables a faster persistent state serializer, requires more RAM.                                                 | Disabled                      | `--fast-state-serializer`                       |

diff --git a/docs/v3/documentation/tvm/changelog/tvm-upgrade-2023-07.md b/docs/v3/documentation/tvm/changelog/tvm-upgrade-2023-07.md
@@ -265,7 +265,11 @@ Flags are similar to `runvmx` in fift:
 - `+32`: return final value of `c5` (actions)
 - `+64`: pop hard gas limit (enabled by ACCEPT) `g_m` from stack
 - `+128`: "isolated gas consumption". Child VM will have a separate set of visited cells and a separate chksgn counter.
-- `+256`: pop integer `r`, return exactly `r` values from the top of the stack (only if `exitcode=0 or 1`; if not enough then `exitcode=stk_und`)
+- `+256`: pop integer `r`, return exactly `r` values from the top:
+      - If RUNVM call successful and r is set, it returns r elements. If r not set - returns all;
+      - if RUNVM successful but there is not enough elements on stack (stack depth less than r) it is considered as exception in child VM, with exit_code=-3 and exit_arg=0 (so 0 is returned as only stack element);
+      - if RUNVM fails with exception - only one element is returned - exit arg (not to be mistaken with exit_code);
+      - in case of OOG, exit_code = -14 and exit_arg is amount of gas.
 
 Gas cost:
 - 66 gas

diff --git a/docs/v3/documentation/tvm/specification/runvm.mdx b/docs/v3/documentation/tvm/specification/runvm.mdx
@@ -0,0 +1,34 @@
+
+# RUNVM Specification
+Currently there is no way for code in TVM to call external untrusted code "in sandbox". In other words, external code always can irreversibly update code, data of contract, or set actions (such as sending all money).
+
+`RUNVM` instruction allows to spawn an independent VM instance, run desired code and get needed data (stack, registers, gas consumption etc) without risks of polluting caller's state. Running arbitrary code in a safe way may be useful for [v4-style plugins](/v3/documentation/smart-contracts/contracts-specs/wallet-contracts#wallet-v4), Tact's `init` style subcontract calculation etc.
+
+| xxxxxxxxxxxxx<br/>Fift syntax | xxxxxxxxxxxxxxxxx<br/>Stack | xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx<br/>Description                                                                                                                           |
+|:-|:-|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `flags RUNVM` | _`x_1 ... x_n n code [r] [c4] [c7] [g_l] [g_m] - x'_1 ... x'_m exitcode [data'] [c4'] [c5] [g_c]`_ | Runs child VM with code `code` and stack `x_1...x_n`. Returns the resulting stack `x'_1...x'_m` and exitcode.<br/>Other arguments and return values are enabled by flags, see below. |
+| `RUNVMX` | _`x_1 ... x_n n code [r] [c4] [c7] [g_l] [g_m] flags - x'_1 ... x'_m exitcode [data'] [c4'] [c5] [g_c]`_ | Same thing, but pops flags from stack.                                                                                                                                               |
+
+Flags are similar to `runvmx` in fift:
+- `+1`: set c3 to code
+- `+2`: push an implicit 0 before running the code
+- `+4`: take `c4` from stack (persistent data), return its final value
+- `+8`: take gas limit `g_l` from stack, return consumed gas `g_c`
+- `+16`: take `c7` from stack (smart-contract context)
+- `+32`: return final value of `c5` (actions)
+- `+64`: pop hard gas limit (enabled by ACCEPT) `g_m` from stack
+- `+128`: "isolated gas consumption". Child VM will have a separate set of visited cells and a separate chksgn counter.
+- `+256`: pop integer `r`, return exactly `r` values from the top:
+- If RUNVM call successful and r is set, it returns r elements. If r not set - returns all;
+- if RUNVM successful but there is not enough elements on stack (stack depth less than r) it is considered as exception in child VM, with exit_code=-3 and exit_arg=0 (so 0 is returned as only stack element);
+- if RUNVM fails with exception - only one element is returned - exit arg (not to be mistaken with exit_code);
+- in case of OOG, exit_code = -14 and exit_arg is amount of gas.
+
+Gas cost:
+- 66 gas
+- 1 gas for every stack element given to the child VM (first 32 are free)
+- 1 gas for every stack element returned from the child VM (first 32 are free)
+
+## See Also
+
+- [TVM Instructions](/v3/documentation/tvm/instructions)
diff --git a/docs/v3/guidelines/nodes/monitoring/performance-monitoring.mdx b/docs/v3/guidelines/nodes/monitoring/performance-monitoring.mdx
@@ -0,0 +1,171 @@
+# Performance Monitoring
+
+## Monitoring TON server performance
+
+
+Tools like `htop`, `iotop`, `iftop`, `dstat`, `nmon` and others are good for measuring real-time performance, but they lack functionalities when it comes to troubleshooting past performance.
+
+This guide recommends and explains how to use the Linux sar (System Activity Report) utility for TON server performance monitoring.
+
+:::tip
+This guideline helps to identify if your server experiences any resource shortage, not if validator-engine performs badly.
+:::
+
+### Installation
+#### SAR Installation
+
+```bash
+sudo apt-get install sysstat
+```
+
+#### Enable automatic statistics gathering
+
+```bash
+sudo sed -i 's/false/true/g' /etc/default/sysstat
+```
+
+#### Enable the service
+
+```bash
+sudo systemctl enable sysstat sysstat-collect.timer sysstat-summary.timer
+```
+
+
+#### Start the service
+
+```bash
+sudo systemctl start sysstat sysstat-collect.timer sysstat-summary.timer
+```
+
+### Usage
+
+By default sar gathers statistics every 10 minutes and shows statistics for the current day, starting at midnight. You can check it by running sar without parameters:
+
+```bash
+sar
+```
+
+
+If you want to see statistics of the previous day or two days before, pass the number as an option:
+
+```bash
+sar -1   # previous day
+sar -2   # two days ago
+```
+
+For the exact date, you should use the f option to point to the sa file of a given day within a month. Thus, for the September 23rd it would be:
+
+```bash
+sar -f /var/log/sysstat/sa23
+```
+
+What sar reports to run and how to read them in order to identify performance issues?
+
+Below is the list of sar commands that can be used to gather different system statistics. You can supplement them with the above options to quickly get the reports for the required date.
+
+
+
+### Memory report
+
+```bash
+sar -rh
+```
+
+Since the TON validator-engine utilizes jemalloc feature it caches a lot of data, this is the reason why sar -rh command most of the time returns a low number in column `%memused`.
+
+At the same time, there always be a high number in the column `kbcached`. For the same reason, you should not worry about the low number of free RAM shown in column `kbmemfree`. The important indicator however is the number that comes from the `%memused` column.
+
+If it goes above 90% you should consider adding more RAM and keep an eye if your validator engine is not stopping abnormally due to OOM (out of memory) reason - the best way to check that is to grep `/var/ton-work/log` file for Signal messages.
+
+
+
+Swap usage
+
+```bash
+sar -Sh
+```
+
+If you notice that swap is used you should consider adding more RAM. The general recommendation from the TON Core team is to have swap disabled.
+
+
+
+### CPU report
+
+```bash
+sar -u
+```
+
+If your server utilizes CPU on average up to 70% (see '%user' column), this should be considered as good.
+
+
+
+### Disk Usage report
+
+```bash
+sar -dh
+```
+
+Watch the '%util' column and react accordingly if it stays above 90% for a particular disk.
+
+
+
+### Network report
+
+```bash
+sar -n DEV -h
+```
+
+or
+
+```bash
+sar -n DEV -h --iface=<interface name>
+```
+
+if you want to filter results by network interface name.
+
+Check out the result of column `%ifutil` - it shows the usage of your interface considering its maximum link speed.
+
+You can check what speed is supported by your NIC by executing the command below:
+
+```bash
+cat /sys/class/net/<interface>/speed
+```
+
+:::info
+This is not the link speed your provider provided you.
+:::
+
+Consider upgrading your link speed if `%ifutil` shows above 70% usage, or columns rxkB/s and txkB/s reporting values close to a bandwidth provided by your provider.
+
+
+
+### Reporting a performance issue
+
+Before reporting any performance issues, please ensure you are fulfilling the minimal requirements for the node. Then execute the following commands:
+
+```bash
+sar -rudh | cat && sar -n DEV -h --iface=eno1 | cat > report_today.txt
+```
+
+For yesterday's report execute:
+
+```bash
+sar -rudh -1 | cat && sar -n DEV -h --iface=eno1 -1 | cat > report_yesterday.txt
+```
+
+Also, stop the TON node and measure your disk IO and network speed.
+
+```bash
+sudo fio --randrepeat=1 --ioengine=io_uring --direct=1 --gtod_reduce=1 --name=test --filename=/var/ton-work/testfile --bs=4096 --iodepth=1 --size=40G --readwrite=randread --numjobs=1 --group_reporting
+```
+
+Look for value at `read: IOPS=` and send it with the report. A value above 10k IOPS should be considered good.
+
+```bash
+curl -s https://raw.githubusercontent.com/sivel/speedtest-cli/master/speedtest.py | python3 -
+```
+Download and Upload speeds above 700 Mbit/s should be regarded as good.
+
+When reporting please send the SAR report as well as IOPS and network speed results to [@mytonctrl_help_bot](https://t.me/mytonctrl_help_bot).
+
+Initial version by [@neodix](https://t.me/neodix) - Ton Core team, September 23, 2024
diff --git a/sidebars/documentation.js b/sidebars/documentation.js
@@ -324,6 +324,13 @@ module.exports = [
     label: 'TVM Instructions',
     href: '/v3/documentation/tvm/instructions',
   },
+  {
+    type: 'category',
+    label: 'TVM Specification',
+    items: [
+      'v3/documentation/tvm/specification/runvm',
+    ],
+  },
   {
     type: 'category',
     label: 'TVM Changelog',

diff --git a/sidebars/guidelines.js b/sidebars/guidelines.js
@@ -227,6 +227,7 @@ module.exports = [
   'v3/guidelines/nodes/custom-overlays',
   'v3/guidelines/nodes/nodes-troubleshooting',
   'v3/guidelines/nodes/node-maintenance-and-security',
+  'v3/guidelines/nodes/monitoring/performance-monitoring',
   'v3/guidelines/nodes/persistent-states',
   'v3/guidelines/nodes/faq',
   {