Parity burning CPU.

[MicahZoltu]

Before filing a new issue, please provide the following information.

I'm running:

• Parity version: Parity//v1.7.0-beta-5f2cabd-20170727/x86_64-windows-msvc/rustc1.18.0
• Operating system: Windows
• And installed: via installer

Your issue description goes here below. Try to include actual vs. expected behavior and steps to reproduce the issue.

---

When I open the Parity UI, the CPU pegs for several seconds. If I close the UI and open it again (without restarting Parity) the CPU pegs again. The same symptoms occur if I open any dApp with the Parity extension active or via the Parity browser.

The exact amount of time the CPU burns for varies, but it can be up to 30 seconds.

For the dApps, I can appreciate that some may be doing a ton of log scanning which causes the problem but I would expect opening the Home page of the the UI to require nothing but a few account state lookups (I have ~6 accounts).

[MicahZoltu]

Actually, in general doing anything in the UI seems to peg my CPU for a while and all of my interactions are really delayed. For example, I just went to execute a contract method. While I was filling in the parameters the UI was pegged (not sure what it was doing). Then when I clicked to send the UI pegged for a while and I had to wait a non-trivial amount of time for the signer to appear. Then the CPU pegged again once I signed.

I don't remember experiencing all of these issues with Parity 1.6.x.

[5chdn]

Could you do me a favor and check your logs while you are using the UI?

Does the number of RPC requests increase significantly after opening the UI?

[MicahZoltu]

After a lot more trial and error and troubleshooting with @tomusdrw I found that rolling back to 1.6.10 did not resolve the issue (including a full chain re-sync since that was required with rollback), and deleting chain and 1.6.10 then installing 1.7.0 and fully syncing also didn't resolve the issue. I also found that it wasn't just a problem when the UI was open, though the UI being open did exacerbate the problem and this appears to be caused by having multiple detached hardware wallets (wiping out my accounts and creating a new single normal account made the UI less impactful when clicking around.

--offline made the problem go away (other than UI unresponsiveness with detached hardware wallets).
--no-ancient-blocks completely resolved my issue, while allowing me to remain synced to the network.

This problem appears to have started after upgrading from 1.6.x to 1.7.0, but unfortunately nothing I did was able to get my computer back into a healthy state (other than --no-ancient-blocks). Parity doesn't appear to be fetching/validating ancient blocks, so it is unclear why this setting would help.

While in this pathologically bad state, My I/O is crazy high, ~100MB/s average (very spiky).

The following screenshot is just parity.exe ui (no other settings). Each tick of the graph is ~1 second, and you gat get an idea for Y-axis scale on the left.

[MicahZoltu]

cc @rphmeier per recommendation from @tomusdrw.

[MicahZoltu]

@5chdn I suspect the tags are wrong, since it turns out this isn't specifically related to the UI. I can edit the description (or you can) if you like to make it more clear what the issue is about.

[5chdn]

You are describing multiple issues here.

1. High CPU usage when fetching ancient blocks (core)
2. Even higher CPU usage when using the wallet, especially with detached hardware wallets (ui)
3. Regarding the insane disk I/O please subscribe here Disk I/O use 100% #6020

Does this sum it up correctly?

Edit, also one of the more recent perfomance issues might be related #6280

[MicahZoltu]

I'm not sure if (3) is unrelated or related, the disk IO didn't bother me but it stood out when I went investigating the CPU burn.

Unfortunately, after running Parity all day today with --no-ancient-blocks the CPU issue returned. :'( I don't know what triggered it, but I'm now into a state where it is burning CPU, then idling for a bit, then burning CPU, then idling.

[rphmeier]

@MicahZoltu does the problem persist once ancient block download is complete?

I actually do expect that ancient block download will lead to high CPU usage, since ethash verification of the header chain is still required. The heavy I/O usage is also because ancient block headers, bodies, and receipts are being written to disk, and the spikes are likely rocksdb compaction. This is something which should be alleviated when block pruning is implemented.

Out of curiosity, how well is your node keeping up with the head of the chain? It could be that the latest blocks are heavy enough to lead your computer to struggle.

[MicahZoltu]

@rphmeier I had previously been running my node locally for months, maintaining sync and without the --no-ancient-blocks flag, so I suspect that I was all caught up with history when the problem started.

I don't notice problems falling behind on block processing when blocks arrive during normal operation, though when it is running at 100% CPU (to the point where other applications on my computer stop behaving correctly) I have seen it fall behind. When Parity is just burning 90% of my CPU for hours on end though, it doesn't fall behind.

[MicahZoltu]

As a minor update, its now a day later and Parity is still running at 100% CPU (default flags at the moment and has been for a while). I have been turning it off periodically so I can actually use my computer, but I try to leave it running overnight at least and when I am doing non-CPU intensive things. Of course, Parity itself has become largely unusable but I would really like to resolve this issue.

Please let me know if there is additional troubleshooting I can do. I recognize that without a repro case it is likely difficult for you all to fix this and since I would like Parity to work again for me I am willing to try to help troubleshoot.

[MicahZoltu]

In a desperate attempt to resolve this issue I bought a new computer (Dell XPS 9650). After fully syncing the chain (see #6372) it is back to burning way more CPU than I would expect while not doing any meaningful work (just keeping up with new blocks). I have no accounts or contracts, outside of one account I created to satisfy the wizard. I have done no customization of Parity or anything notable to this computer (Windows 10 Pro, Docker, Node, Chrome).

According to process explorer, it is burning ~15% (10%-30% range) of CPU at all times while in "keep the chain synced mode". Disk IO ranges from 10MB/s to 150MB/sec.

Changing the mode to "wake to sync" doesn't appear to noticeably change the utilization profile.

Here is a utilization graph of just parity.exe, part way through I switched from wake to sync to continuous sync and as you can see there is no obvious change:

With the problem reproducing on a brand new computer, I have concluded that its not just a problem with my setup/computer/install or a problem with my usage of Parity (accounts, contracts, etc.). The problem isn't as bad on this new computer as it was on the previous one, but I would expect lower background utilization than I'm seeing on a modern computer.

[MicahZoltu]

After many more hours of sitting idle on the computer, it appears to have calmed down to acceptable levels. Perhaps the issue is simply that the UI doesn't give enough information about what is happening (both for troubleshooting, and to make me not freak out)? Presumably, something changed over the past few hours but the UI gives me no indication that it is in any different of a state now than it was several hours ago. Previously it has been mentioned that there was some process that occurs after a sync, on my new computer I suspect that was probably the case. Is there an issue to have the UI convey this information to the user? If not, should I create one?

Note: Parity is still completely unusable on my original computer.

[5chdn]

In a desperate attempt to resolve this issue I bought a new computer

🤣

After many more hours of sitting idle on the computer, it appears to have calmed down to acceptable levels. Perhaps the issue is simply that the UI doesn't give enough information about what is happening (both for troubleshooting, and to make me not freak out)?

Go to the status tab, scroll to the bottom and follow the logs by clicking the play button. After a warp sync it fetches ancient blocks, this process takes several hours and fits very well your description above.

[arkpar]

@MicahZoltu After the initial warp sync is complete there's a background process that downloads and validates historical blocks. This process takes a few hours and is CPU demanding. Currently there's progress indication in the console output/logs, but not in the UI.

[5chdn]

I'm tempted to close this as working as desired. This is not really a Parity issue but an Ethereum issue. Your client is processing 400k transactions per day currently, even if it's not sync'ing, that's 5 per second.

Regarding the high IO, there is room for optimization regarding the database layer, but there is a separate ticket open. #6280 #6020

Regarding the accounts and dapps issue, that's something we are aware of. #6387

[MicahZoltu]

@5chdn I won't fight closure of this issue since it doesn't seem like anyone has any ideas for what I can do to further troubleshoot, but I would argue pretty strongly that "working as intended" is not correct (perhaps could-not-reproduce instead?). When I say Parity is burning CPU, I mean 100% CPU 24/7. My computer is unusable, to the point where the OS itself even locks up sometimes (I thought Windows had solved this and made it so the OS always had priority, I'm guessing it is the IO that is blocking in the Kernel).

On my new computer, Parity uses a medium amount of CPU initially while syncing and then archiving (~15%-30%), but then drops down to background idle of ~1% utilization (with UI closed). To me the behavior on my new computer is "working as intended" and the behavior on my old computer (which is only ~6 months old) is definitely not "working as intended".

On my old computer I have have quit using Parity, and if it weren't for having just bought a new computer I would have switched to Geth or MetaMask or something (not sure what exactly).

[5chdn]

My computer is unusable, to the point where the OS itself even locks up sometimes (I thought Windows had solved this and made it so the OS always had priority, I'm guessing it is the IO that is blocking in the Kernel).

Yeah, I'm getting this pretty much and therefore referred to the open IO tickets.

If you say it's not IO but CPU and your computer does that 24/7, I would have to ask you for logs to figure out what's going on (Status tab at the bottom).

Could you also tell me, did you have a lot of accounts on your old computer?

[MicahZoltu]

I have 5 accounts (I removed my hardware wallets), 2 addresses and 2 contracts. I used to have more, but I have deleted most of them to try to troubleshoot.

I have previously watched the logs when in this state and nothing appears out of the ordinary. Is there a way in the UI to get more detailed logs than the default?

I'll turn Parity back on on the old computer with log capture enabled and post the logs here (I don't expect anything though based on my past experience).

[MicahZoltu]

Things were actually looking healthy for a bit, then around 1:42 CPU jumped to 100% for 4.5 minutes, finally stabilizing at ~1:44:30.

[8/28/2017, 1:46:24 PM] 25/25 peers 3 MiB chain 79 MiB db 0 bytes queue 3 MiB sync RPC: 1 conn, 4 req/s, 306 µs
[8/28/2017, 1:45:49 PM] 25/25 peers 6 MiB chain 79 MiB db 0 bytes queue 3 MiB sync RPC: 1 conn, 4 req/s, 140 µs
[8/28/2017, 1:45:14 PM] 25/25 peers 7 MiB chain 79 MiB db 0 bytes queue 3 MiB sync RPC: 1 conn, 7 req/s, 85 µs
[8/28/2017, 1:44:54 PM] Imported #4213996 4607…4881 (109 txs, 6.56 Mgas, 50.67 ms, 19.47 KiB)
[8/28/2017, 1:44:46 PM] Imported #4213995 ff57…430a (98 txs, 6.68 Mgas, 45.73 ms, 18.91 KiB)
[8/28/2017, 1:44:39 PM] 25/25 peers 7 MiB chain 79 MiB db 0 bytes queue 3 MiB sync RPC: 1 conn, 3 req/s, 11955852 µs[8/28/2017, 1:44:14 PM] Imported #4213994 21f2…4230 (106 txs, 6.64 Mgas, 89.09 ms, 21.06 KiB)
[8/28/2017, 1:44:04 PM] 25/25 peers 7 MiB chain 78 MiB db 0 bytes queue 3 MiB sync RPC: 1 conn, 8 req/s, 596 µs
[8/28/2017, 1:43:45 PM] Imported #4213993 b8d7…13c4 (32 txs, 1.48 Mgas, 34.23 ms, 5.49 KiB)
[8/28/2017, 1:43:29 PM] 24/25 peers 6 MiB chain 78 MiB db 0 bytes queue 3 MiB sync RPC: 1 conn, 3 req/s, 148 µs
[8/28/2017, 1:43:08 PM] Imported #4213992 63a8…c7ee (70 txs, 6.60 Mgas, 49.67 ms, 14.28 KiB)
[8/28/2017, 1:42:55 PM] Imported #4213991 556b…d5c3 (67 txs, 6.67 Mgas, 113.40 ms, 17.55 KiB)
[8/28/2017, 1:42:54 PM] 23/25 peers 96 KiB chain 78 MiB db 0 bytes queue 3 MiB sync RPC: 1 conn, 358 req/s, 14671972 µs
[8/28/2017, 1:42:46 PM] Imported #4213990 aa41…88d2 (144 txs, 6.69 Mgas, 108.74 ms, 20.40 KiB)
[8/28/2017, 1:42:33 PM] Imported #4213989 de5d…e1f8 (190 txs, 6.71 Mgas, 157.30 ms, 28.53 KiB)
[8/28/2017, 1:42:25 PM] Imported #4213988 4618…5377 (22 txs, 0.48 Mgas, 15.70 ms, 4.00 KiB)
[8/28/2017, 1:42:24 PM] 24/25 peers 7 MiB chain 78 MiB db 0 bytes queue 3 MiB sync RPC: 0 conn, 0 req/s, 210 µs
[8/28/2017, 1:41:53 PM] Imported #4213987 cef8…f138 (13 txs, 0.55 Mgas, 16.34 ms, 2.17 KiB) + another 1 block(s) containing 157 tx(s)
[8/28/2017, 1:41:49 PM] 24/25 peers 8 MiB chain 78 MiB db 0 bytes queue 3 MiB sync RPC: 0 conn, 0 req/s, 210 µs
[8/28/2017, 1:41:48 PM] Imported #4213985 d4c1…be05 (145 txs, 5.12 Mgas, 72.64 ms, 21.68 KiB)
[8/28/2017, 1:41:44 PM] Imported #4213986 5a1d…7279 (105 txs, 3.67 Mgas, 42.80 ms, 15.13 KiB)
[8/28/2017, 1:41:37 PM] Imported #4213985 7e67…2174 (156 txs, 6.68 Mgas, 92.17 ms, 27.42 KiB)
[8/28/2017, 1:41:18 PM] Imported #4213984 8111…cb17 (75 txs, 6.70 Mgas, 24.91 ms, 13.44 KiB)
[8/28/2017, 1:41:14 PM] 25/25 peers 6 MiB chain 79 MiB db 0 bytes queue 3 MiB sync RPC: 0 conn, 0 req/s, 210 µs
[8/28/2017, 1:40:39 PM] 24/25 peers 7 MiB chain 79 MiB db 0 bytes queue 3 MiB sync RPC: 1 conn, 8 req/s, 297 µs
[8/28/2017, 1:40:09 PM] 22/25 peers 6 MiB chain 79 MiB db 0 bytes queue 3 MiB sync RPC: 1 conn, 5 req/s, 9116613 µs
[8/28/2017, 1:39:59 PM] Imported #4213983 84f0…468f (70 txs, 2.65 Mgas, 51.13 ms, 10.09 KiB)
[8/28/2017, 1:39:54 PM] Imported #4213982 f980…fc33 (90 txs, 6.63 Mgas, 57.96 ms, 19.41 KiB)
[8/28/2017, 1:39:39 PM] 22/25 peers 6 MiB chain 78 MiB db 0 bytes queue 3 MiB sync RPC: 1 conn, 7 req/s, 263 µs
[8/28/2017, 1:39:22 PM] Imported #4213981 3f83…c393 (65 txs, 6.62 Mgas, 46.17 ms, 11.07 KiB)
[8/28/2017, 1:39:09 PM] 22/25 peers 6 MiB chain 78 MiB db 0 bytes queue 3 MiB sync RPC: 1 conn, 10 req/s, 145 µs

[5chdn]

Logs are looking good. I'm pretty sure the issues you are describing is related to #6387. We are preparing a fix currently. I would like you to stay tuned for the patch to test it on your machine.

[dev-dan]

I don't know if related, but when I have the parity signer extension open, my parity 1.7 parity.exe CPU usage shoots to 100% with small drops down to 50% but mostly staying at 100%. This occurs until I close the signer.

[MicahZoltu]

I don't use the signer extension so that isn't the same symptoms as I have, though perhaps similar root cause?