After running for a period of time, the memory usage is very high, and it will recover after restarting. happens very often

[atoken-ops]

• OpenEthereum version: 3.0.1
• Operating system: Linux
• Installation: homebrew / one-line installer
• Fully synchronized: yes
• Network: ethereum
• Restarted: yes

[Joter271]

Have same issue. Windows client.

[mr-older]

Data from two nodes on one machine, debian 9.11:

ETH (running for months) consumes 11/3.1/0 Gb (virtual / resident / shared) - Parity-Ethereum/v2.6.8-beta-9bf6ed8-20191231/, warp = false, light = false

ETC (couple of days & full-synced with ancient blocks today) got 21/5/0 Gb - OpenEthereum/v3.0.1-stable-8ca8089-20200601/, warp = true, light = false

Virtual memory used seems to be too large even for ETC. There is a 1Gb of swap (earlier with ETC on parity was much less). Even login to OS became very slow from the same network after migrating ETC from Parity to OpenEthereum.

[atoken-ops]

Anyone？

[Joter271]

I think this issue is related to: https://github.com/openethereum/openethereum/issues/11737

[atoken-ops]

how fix it.nobody care?

[adria0]

@atoken-ops, we are working on this issue.

[atoken-ops]

now,the client stop import new block.reboot can not fix .how fix?
2020-06-16 12:06:50 Syncing #10269517 0xd310…24a7 0.00 blk/s 0.0 tx/s 0.0 Mgas/s 0+ 1523 Qed #10271040 15/25 peers 2 MiB chain 28 GiB db 248 MiB queue 141 MiB sync RPC: 0 conn, 0 req/s, 8837 µs
2020-06-16 12:06:58 Syncing #10269517 0xd310…24a7 0.00 blk/s 0.0 tx/s 0.0 Mgas/s 0+ 1523 Qed #10271040 16/25 peers 3 MiB chain 28 GiB db 248 MiB queue 141 MiB sync RPC: 0 conn, 0 req/s, 8837 µs
2020-06-16 12:07:06 Syncing #10269517 0xd310…24a7 0.00 blk/s 0.0 tx/s 0.0 Mgas/s 0+ 1523 Qed #10271040 16/25 peers 3 MiB chain 28 GiB db 248 MiB queue 141 MiB sync RPC: 0 conn, 0 req/s, 8837 µs
2020-06-16 12:07:14 Syncing #10269517 0xd310…24a7 0.00 blk/s 0.0 tx/s 0.0 Mgas/s 0+ 1523 Qed #10271040 16/25 peers 3 MiB chain 28 GiB db 248 MiB queue 141 MiB sync RPC: 0 conn, 0 req/s, 8837 µs
2020-06-16 12:07:22 Syncing #10269517 0xd310…24a7 0.00 blk/s 0.0 tx/s 0.0 Mgas/s 0+ 1523 Qed #10271040 16/25 peers 3 MiB chain 28 GiB db 248 MiB queue 141 MiB sync RPC: 0 conn, 0 req/s, 8837 µs
2020-06-16 12:07:25 personal_unlockAccount is deprecated and will be removed in future versions: Account management is being phased out see #9997 for alternatives.
2020-06-16 12:07:30 Syncing #10269517 0xd310…24a7 0.00 blk/s 0.0 tx/s 0.0 Mgas/s 0+ 1523 Qed #10271040 17/25 peers 3 MiB chain 28 GiB db 248 MiB queue 141 MiB sync RPC: 0 conn, 0 req/s, 8837 µs
2020-06-16 12:07:39 Syncing #10269517 0xd310…24a7 0.00 blk/s 0.0 tx/s 0.0 Mgas/s 0+ 1523 Qed #10271040 17/25 peers 3 MiB chain 28 GiB db 248 MiB queue 141 MiB sync RPC: 0 conn, 0 req/s, 8837 µs
2020-06-16 12:07:47 Syncing #10269517 0xd310…24a7 0.00 blk/s 0.0 tx/s 0.0 Mgas/s 0+ 1523 Qed #10271040 16/25 peers 3 MiB chain 28 GiB db 248 MiB queue 141 MiB sync RPC: 0 conn, 0 req/s, 8837 µs
2020-06-16 12:07:55 Syncing #10269517 0xd310…24a7 0.00 blk/s 0.0 tx/s 0.0 Mgas/s 0+ 1523 Qed #10271040 16/25 peers 3 MiB chain 28 GiB db 248 MiB queue 141 MiB sync RPC: 0 conn, 0 req/s, 8837 µs
2020-06-16 12:08:03 Syncing #10269517 0xd310…24a7 0.00 blk/s 0.0 tx/s 0.0 Mgas/s 0+ 1523 Qed #10271040 16/25 peers 3 MiB chain 28 GiB db 248 MiB queue 141 MiB sync RPC: 0 conn, 0 req/s, 8837 µs
2020-06-16 12:08:11 Syncing #10269517 0xd310…24a7 0.00 blk/s 0.0 tx/s 0.0 Mgas/s 0+ 1523 Qed #10271040 16/25 peers 3 MiB chain 28 GiB db 248 MiB queue 141 MiB sync RPC: 0 conn, 0 req/s, 8837 µs
2020-06-16 12:08:19 Syncing #10269517 0xd310…24a7 0.00 blk/s 0.0 tx/s 0.0 Mgas/s 0+ 1523 Qed #10271040 17/25 peers 3 MiB chain 28 GiB db 248 MiB queue 141 MiB sync RPC: 0 conn, 0 req/s, 8837 µs

[rakita]

Hello, we did some investigation and it seems that when we are taking the snapshot pruning is disabled for the obvious reason we don't want to remove blocks needed for the snapshot (See this #11178 for discussion). Because the default memory model is OverlayRecentDB memory, when we did a test we saw that memory after snapshot jumps to 13gb for you it seems it is 28 GiB, and jump is somehow expected but in the initial discussion, it was 500-1800mb.
We will do two things first one is that we will disable snapshotting by default in the next release.
The second thing that we notice is that after snapshot finishes OverlayRecentDB ->MemoryDB -> HashMap does not do shrink_to_fit and that big HashMap stays big. I am currently testing this branch that should help.