Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add docs about increasing TTL #253

Open
2 tasks
Tracked by #392
6r1d opened this issue Feb 6, 2023 · 5 comments
Open
2 tasks
Tracked by #392

Add docs about increasing TTL #253

6r1d opened this issue Feb 6, 2023 · 5 comments
Assignees
Labels
I-peer Documentation specific to Iroha peers L-config Configuration, customization

Comments

@6r1d
Copy link
Contributor

6r1d commented Feb 6, 2023

I've heart that @Mingela reported the symptoms like these:

  • lots of “this peer is faulty” in the logs
  • "view change loop"

According to @appetrosyan, this could be fixed with changing the TTL / timeout configuration parameters.
Also, it should be possible to mitigate the problem by using more performant hardware.
As far as I understand, those apply to WASM, but I could be wrong.

For starters, I'd like to get the example logs from @Mingela so I can update the documentation troubleshooting section.


General approach

  • Extend the troubleshooting with a section describing a condition where the blocks aren't produced, cpu consuption approaches the 100% and there are lots “This peer is faulty” messages
  • point out that the block_time and commit_time parameters should be increased
@6r1d
Copy link
Contributor Author

6r1d commented Feb 6, 2023

According to @SamHSmith, I have to address those options:

"BLOCK_TIME_MS": 1000,
"COMMIT_TIME_LIMIT_MS": 2000,

(I'd focus on these, but include a caution about transactions expiring.)

You increase these values on all the peers. The block time is the time taken to create a block and the commit time the time to reach consensus.

If the values are higher then the amount of messages is lower and things take longer.

One thing to remember is that you might also have to adjust the "time to live" or ttl on the transactions submitted. If the network is going to act slower you must make sure your transactions are not expiring.

Because if the network is going to act slower you must make sure your transactions are not expiring.

This one on the queue

"TRANSACTION_TIME_TO_LIVE_MS": 86400000,

Although this default setting of a day is probably fine.

@6r1d
Copy link
Contributor Author

6r1d commented Feb 6, 2023

@SamHSmith's comment on "This peer is faulty. Incoming messages have to be dropped due to low processing speed." messages:

That message indicates that the sumeragi loop is not able to process incoming messages fast enough, causing them to pile up. The solution is either to have more CPU so that it can run faster, or you need to reduce the amount of messages. And to reduce the amount of messages you have to increase the block time and commit time. You "virtually" speed up sumeragi by slowing down the operation of the network.

@6r1d
Copy link
Contributor Author

6r1d commented Feb 6, 2023

I'm also adding a cleaned-up configuration @Mingela provided to extend the doc.

{
          "BLOCK_SYNC": {
              "ACTOR_CHANNEL_CAPACITY": 100,
              "BLOCK_BATCH_SIZE": 4,
              "GOSSIP_PERIOD_MS": 30000
          },
          "DISABLE_PANIC_TERMINAL_COLORS": false,
          "GENESIS": {
            "ACCOUNT_PRIVATE_KEY": {
                "digest_function": "ed25519",
                "payload": <deleted>
            },
            "ACCOUNT_PUBLIC_KEY": <deleted>
            "GENESIS_SUBMISSION_DELAY_MS": 300000,
            "WAIT_FOR_PEERS_RETRY_COUNT_LIMIT": 500,
            "WAIT_FOR_PEERS_RETRY_PERIOD_MS": 5000
          },
          "KURA": {
              "ACTOR_CHANNEL_CAPACITY": 100,
              "BLOCKS_PER_STORAGE_FILE": 1000,
              "BLOCK_STORE_PATH": "/iroha2/data/blocks",
              "INIT_MODE": "strict",
              "DEBUG_OUTPUT_NEW_BLOCKS": true
          },
          "LOGGER": {
              "COMPACT_MODE": false,
              "LOG_FILE_PATH": "/iroha2/logs.json",
              "MAX_LOG_LEVEL": "ERROR",
              "TELEMETRY_CAPACITY": 1000,
              "TERMINAL_COLORS": true
          },
          "NETWORK": {
              "ACTOR_CHANNEL_CAPACITY": 100
          },
          "PRIVATE_KEY": {
            "digest_function": "ed25519",
            "payload": <deleted>
          },
          "PUBLIC_KEY": <deleted>
          "QUEUE": {
              "FUTURE_THRESHOLD_MS": 60000,
              "MAXIMUM_TRANSACTIONS_IN_BLOCK": 8192,
              "MAXIMUM_TRANSACTIONS_IN_QUEUE": 65536,
              "TRANSACTION_TIME_TO_LIVE_MS": 604800000
          },
          "SUMERAGI": {
              "KEY_PAIR": null,
              "ACTOR_CHANNEL_CAPACITY": 100,
              "BLOCK_TIME_MS": 12000,
              "COMMIT_TIME_LIMIT_MS": 20000,
              "GOSSIP_BATCH_SIZE": 500,
              "GOSSIP_PERIOD_MS": 10000,
              "PEER_ID": null,
              "TRANSACTION_LIMITS": {
                  "max_instruction_number": 4096,
                  "max_wasm_size_bytes": 4194304
              },
              "TRUSTED_PEERS": [
                <deleted>
              ],
              "TX_RECEIPT_TIME_LIMIT_MS": 10000
          },
          "TELEMETRY": {
              "FILE": null,
              "MAX_RETRY_DELAY_EXPONENT": 4,
              "MIN_RETRY_PERIOD": 1,
              "NAME": null,
              "URL": null
          },
          "TORII": {
              "API_URL": "0.0.0.0:8080",
              "MAX_CONTENT_LEN": 16384000,
              "MAX_TRANSACTION_SIZE": 32768,
              "P2P_ADDR": "0.0.0.0:1337",
              "TELEMETRY_URL": "0.0.0.0:8180"
          },
          "WSV": {
              "ACCOUNT_METADATA_LIMITS": {
                  "max_entry_byte_size": 4096,
                  "max_len": 1048576
              },
              "ASSET_DEFINITION_METADATA_LIMITS": {
                  "max_entry_byte_size": 4096,
                  "max_len": 1048576
              },
              "ASSET_METADATA_LIMITS": {
                  "max_entry_byte_size": 4096,
                  "max_len": 1048576
              },
              "DOMAIN_METADATA_LIMITS": {
                  "max_entry_byte_size": 4096,
                  "max_len": 1048576
              },
              "IDENT_LENGTH_LIMITS": {
                  "max": 128,
                  "min": 1
              },
              "WASM_RUNTIME_CONFIG": {
                  "FUEL_LIMIT": 900000000000,
                  "MAX_MEMORY": 524288000
              }
          }
      }

@6r1d 6r1d self-assigned this Feb 6, 2023
@outoftardis
Copy link
Contributor

will be partly addressed via config rfc

@0x009922
Copy link
Contributor

Yes, will be partially addressed by the configuration reference overhaul:

@nxsaken nxsaken added L-config Configuration, customization I-peer Documentation specific to Iroha peers and removed iroha2 labels May 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
I-peer Documentation specific to Iroha peers L-config Configuration, customization
Projects
None yet
Development

No branches or pull requests

4 participants