Server got "java.net.SocketException: Broken pipe" #6664
Comments
|
hi @kwoxer, This means just that a client close the connection while the server was sending data, it does not cause any harm, you can ignore this. Regards |
|
Mhh well, it seems to be online just for some hours. Offline since 2 days as it seems. The log is also very empty: I gonna let the server open via putty, maybe there is some more info when it crashes. Or is there another way to investigate more about this? Btw this is the server.sh log when I restart the server: Let's see when it crashes... |
|
Hi @kwoxer, Ok, let us know if you have further problems. Regards |
|
So today it some how stopped again. But sadly nothing in the logs =/ This is what I have in the open server.sh window via ssh shell. Accessed via screen: As you see at the end I got: I gonna start the server.sh now again. But as you have seen, it ran again 2-3 days. So it is repreducable! Any ideas? |
|
@kwoxer I suppose that your close system application because big usage of system resources like memory so you should decrease memory settings to avoid such problems in future. |
|
Also I do suggest to update to 2.2.10 version of database. |
|
Okay. So when again "killed" I try to set lower memory stuff. Is half of current okay? Or trying to give a bit more? Currently the server is still running. :) |
|
Well not the half, I think about 3/4 of current memory, do not forget to decrease disk cache size and memory chunk size too. |
|
So the last run worked about 3 days. Now crashed. I gonna try it with 3/4 memory now. So instead of: now: |
|
Well again crashed/KILLED after 3 days. Also with lower settings: Now I gonna try with 1/2 memory. Please tell me if you see something weird. Thanks: |
|
Okay, OrientDB still running. Already 4 days. I'm checking now every day. I'm updating the state here: |
|
@kwoxer any news on this? |
|
Server still running. As you see I updated my last post here. Will repost new config here in the case the server is crashing again. But anyway it would be nice to know why it now seems to run well. As said I'm now on 50% memory. But maybe there is one settings of the many that was the main reason. I don't know. :) |
|
Ok just for you info. This morning I checked the state. All was fine. But checked right now, and it's down. So today the server crashed. This means, it ran 7 days with the 1/2 memory settings. I'm going down to 1/3 memory now. But could someone give me tips what to especially change? |
|
Cool and here is something I got from the log, maybe it helps: |
|
With 1/3 memory. Current state:
|
|
Ok, 12 days now. Still too less :( So again the last lines: What can I try next? Which setting might be the real reason? |
|
@kwoxer is that client log right ? Could you send server log too ? |
|
Well this is all what I can read from the log: |
|
This is really bad. It's hard to test at already lower values. So lowering all is making it better, we can say! Now I try to lower specific values, to test which have a real influence on the server runtime. So instead of: I now take: So I just changed this line Let's see how long it works. |
|
@maggiolo00 could you look on log output ? |
|
Server still online. So it's already longer than usual. :) |
|
Offline since 12.11. So from 2 days it's now 5 days. So it seems like buffersize und chunk size is something that increases the time. :) I keep on my tests. |
|
Now: |
|
Now down. So 6-7 days. Puh I no idea. This is like random I believe... |
|
hi @kwoxer, for monitor the memory you can run a simple cron every minute and log the process status. ex script to get a memory dump if memory exceed 300MB the I hope is a linux system, otherwise no idea :) |
|
Sure I have an Ubuntu running. I now have: Hopefull this helps you to help me :) |
|
hi @kwoxer, Be aware that for us probably a jmap is more usefull ;). Bye |
|
Damn. I cannot get it to work. The file isn't created with your commands. So I tried a bit around but was not able to find the correct way of using jmap. Maybe something like:
Please help me out with the correct jmap command. Cron will run it every 1 min. |
|
hi @kwoxer, Umm ok, if you are running it with chron double check that everything has te right permission on the file system Regard |
|
Nono, when I run the commands on my own, I for example get an issue with So what would be the command without |
|
Mhh okay so no answer here :( I tried to use jmap now and played a bit around. But somehow I fail to give jmap the process ID. I have: How can I give jmap to ID. It's a shell question. :) |
|
hi @kwoxer, what do you get just with "ps aux | grep orient | grep -v grep " ? Regards |
|
ps aux | grep orient | grep -v grep | awk {print $2} gives me the correct ID of the orientdb process. I think that answers your question :) |
|
Still waiting for help =/ |
|
hi @kwoxer, i think the problem was the loation of the file, can you try: Regards |
|
jmap is not usefull right? Because installing jstack is ab it tricky. Here is what comes from jmap: Is this usefull? |
|
Ahh ok jstack is now working. So this is when orientdb is running: "Attach Listener" #17 daemon prio=9 os_prio=0 tid=0x00007fcfb8001000 nid=0x2193 runnable [0x0000000000000000] "DestroyJavaVM" #16 prio=5 os_prio=0 tid=0x00007fcfdc00d000 nid=0x1fe1 waiting on condition [0x0000000000000000] "OrientDB ONetworkProtocolHttpDb listen at 0.0.0.0:2480-2490" #14 prio=5 os_prio=0 tid=0x00007fcfdc886000 nid=0x1fec runnable [0x00007fcfaee42000] "OrientDB ONetworkProtocolBinary listen at 0.0.0.0:2424-2430" #12 prio=5 os_prio=0 tid=0x00007fcfdc872000 nid=0x1feb runnable [0x00007fcfaef43000] "Timer-0" #9 daemon prio=5 os_prio=0 tid=0x00007fcfdc65b000 nid=0x1fea in Object.wait() [0x00007fcfafa7a000] "Service Thread" #7 daemon prio=9 os_prio=0 tid=0x00007fcfdc0ac000 nid=0x1fe8 runnable [0x0000000000000000] "C1 CompilerThread1" #6 daemon prio=9 os_prio=0 tid=0x00007fcfdc0a9000 nid=0x1fe7 waiting on condition [0x0000000000000000] "C2 CompilerThread0" #5 daemon prio=9 os_prio=0 tid=0x00007fcfdc0a7000 nid=0x1fe6 waiting on condition [0x0000000000000000] "Signal Dispatcher" #4 daemon prio=9 os_prio=0 tid=0x00007fcfdc0a4800 nid=0x1fe5 runnable [0x0000000000000000] "Finalizer" #3 daemon prio=8 os_prio=0 tid=0x00007fcfdc07d000 nid=0x1fe4 in Object.wait() [0x00007fcfccffe000] "Reference Handler" #2 daemon prio=10 os_prio=0 tid=0x00007fcfdc078800 nid=0x1fe3 in Object.wait() [0x00007fcfe0107000] "VM Thread" os_prio=0 tid=0x00007fcfdc071000 nid=0x1fe2 runnable "VM Periodic Task Thread" os_prio=0 tid=0x00007fcfdc0cb800 nid=0x1fe9 waiting on condition JNI global references: 1606 |
|
1 * * * * if [ is the crontab script. But somehow it did not write. Why is the cron not writing to this file? Is the issue with crontab or with the script? |
|
Please help me guys =/ Could you maybe setup a Amazon Web Service with 512 MB RAM and test it with me? Thank you. |
|
@kwoxer It may be related to the fact that ODB consumes more memory than we expect I will keep you posted when we solve given issue. I suppose it will take week or two. |
|
Yes it's still the memory as single issue. Great. Hopefully you can help me then. Thank you |
|
And how is it going, 1 month later? :) |
|
Something new? I just updated the server to latest software. And now started server again. Maybe now it works a bit longer... |
|
👍 |
|
hi @kwoxer, @parimal-padture, Some issue on the http layer have been fixed in 2.2.20, I think also this issue has been solved, you can download last hotfix 2.2.21 regards |
|
Still not working with the latest release. But I was able to use a workaround now. I'm doing it with https://stackoverflow.com/questions/696839/how-do-i-write-a-bash-script-to-restart-a-process-if-it-dies So until sh server.sh; do is working perfectly for me. So shall this here be closed? I mean it's still an issue but with automatically restarting the server all is fine for me and it's like a proper solution. |
|
Hi @kwoxer, This is a while ago, I would suggest to update to a more recent hotfix, we do not have anyone reporting problem like this in recent hotfixes. Closing how you mentioned. |
|
Hey guys, I had the same issue. Realized that I did not pass this into the JVM: System.setProperty("javax.net.ssl.keyStore", "src/main/resources/certificates/YOUR_CERT) |
OrientDB Version, operating system, or hardware.
Operating System
Expected behavior and actual behavior
Expected : Server should run well for ever
Actual: Server got this exception after 2 days running well
Steps to reproduce the problem
Error message:
The text was updated successfully, but these errors were encountered: