etcd high cpu usage #1742
Labels
Milestone
Comments
|
Output from [
{
"name": "508913e0694b4354be7ae5fd0c044e14",
"state": "follower",
"clientURL": "http://172.17.8.101:4001",
"peerURL": "http://172.17.8.101:7001"
},
{
"name": "7762f6e808934d57b0092100cf600743",
"state": "follower",
"clientURL": "http://172.17.8.160:4001",
"peerURL": "http://172.17.8.160:7001"
},
{
"name": "9632e08e866d4a769436ab7c2fc70e6b",
"state": "leader",
"clientURL": "http://172.17.8.150:4001",
"peerURL": "http://172.17.8.150:7001"
}
][
{
"name": "508913e0694b4354be7ae5fd0c044e14",
"state": "follower",
"clientURL": "http://172.17.8.101:4001",
"peerURL": "http://172.17.8.101:7001"
},
{
"name": "7762f6e808934d57b0092100cf600743",
"state": "leader",
"clientURL": "http://172.17.8.160:4001",
"peerURL": "http://172.17.8.160:7001"
},
{
"name": "9632e08e866d4a769436ab7c2fc70e6b",
"state": "follower",
"clientURL": "http://172.17.8.150:4001",
"peerURL": "http://172.17.8.150:7001"
}
][
{
"name": "508913e0694b4354be7ae5fd0c044e14",
"state": "leader",
"clientURL": "http://172.17.8.101:4001",
"peerURL": "http://172.17.8.101:7001"
},
{
"name": "7762f6e808934d57b0092100cf600743",
"state": "follower",
"clientURL": "http://172.17.8.160:4001",
"peerURL": "http://172.17.8.160:7001"
},
{
"name": "9632e08e866d4a769436ab7c2fc70e6b",
"state": "follower",
"clientURL": "http://172.17.8.150:4001",
"peerURL": "http://172.17.8.150:7001"
}
] |
|
Ok, this might be related to: #868 |
|
@TheHippo Are you seeing thrashing? |
|
@philips What exactly do you mean with trashing? Are there any more information I could provide to help your solve this issue? |
|
I uploaded a log file here. Last 40 lines: |
|
I found a way to "fix" this issue. I added the following lines to my write_files:
- path: /run/systemd/system/etcd.service.d/30-timing.conf
permissions: 420
content: |
[Service]
# Peer Env Vars
Environment=ETCD_PEER_HEARTBEAT_INTERVAL=150
Environment=ETCD_PEER_ELECTION_TIMEOUT=750 |
|
I'm also seeing similar behavior on 3 test nodes. Happy to provide details if needed. |
It is good to have more details. @nakker Open new issues with details if you want to report it. :) |
|
Feel free to reopen it if this is still a problem. |
|
I do have the same problem. The "fix" just makes the coreos cluster more stable (i.e. not electing new master every second), but still the etcd daemon takes 80-90% of one core
Currently the server is brand new - without any docker instances: |
|
@mlazarov We have moved to 2.0.5, and gonna release it on CoreOS very soon. Please give it a try, and it should be much stable. |
|
Same issue on |
|
@oguzbilgic Open an new issue if this problem still exists. |
|
I'm experiencing this problem. Had lots of logging about elections, so I wiped the data dir, thinking perhaps etcd keeps trying to reelect with old, non-extant peers. Now the logging has stopped, but etcd is still at 95% CPU. Not sure why a new issue needs to be opened when this issue and #868 were closed without any concrete resolution steps. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I'm currently trying out CoreOS with 3 virtual machines via Virtual Box and found out, that the cpu usage for
etcdgrows over time. (From 35% to 90% over the course of half an hour.) This seem to be happening regardless of the CoreOS update channel I use.I'm not an expert with etcd, but I'm willing to provide addional data if this helps solve this issue.
The text was updated successfully, but these errors were encountered: