HBASE-24396 : RetryCounter#sleepUntilNextRetry and ThrottledInputStre… #1765
Conversation
…am#throttle should use uninterrupted sleep
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
| } catch (InterruptedException e) { | ||
| throw new InterruptedIOException("Thread aborted"); | ||
| } | ||
| Uninterruptibles.sleepUninterruptibly(sleepTime, TimeUnit.MILLISECONDS); |
There was a problem hiding this comment.
Do we need to this jira? When during the throttle sleep, if we interrupt this thread, it would have come out of sleep by throwing an IOE as per the current code. But a call to sleepUninterruptibly will make sure the thread is in sleep state for that much time. The interrupt might be for a genuine case to stop the running thread. Now this change will make it such that even if been interrupted, the thread will still continue to be executed!
There was a problem hiding this comment.
Same here, throttle()'s purpose should ideally be to throttle without any interruption.
| } catch (InterruptedException e) { | ||
| throw new RuntimeException(e); | ||
| } | ||
| retryCounter.sleepUntilNextRetry(); |
There was a problem hiding this comment.
Here also we are changing the behave. Previously throw RTE when interrupted. But now this is been changed Main thing is even if interrupted, the RetryCounter will make sure the thread been slept for the specified time (Which might not be really wanted some times)
There was a problem hiding this comment.
I believe the purpose of retryCounter.sleepUntilNextRetry() should be uninterrupted sleep because RetryCounter is mainly being used by retries with sleeps and retries with different backoff policies. In such scenario, RetryCounter being a library should not ideally throw InterruptedException even if sleep is interrupted because it is being retried by clients to achieve certain tasks.
|
Both RetryCounter and ThrottledInputStream are Private.IA being used by clients to retry/throttle with sleep and IMHO, the sleep used internally by these libraries should be uninterruptible and clients should not worry about handling InterruptedException because clients will require smooth retries with backoff. |
| @@ -130,11 +130,7 @@ void triggerFlushInPrimaryRegion(final HRegion region) throws IOException { | |||
| ServerRegionReplicaUtil.getRegionInfoForDefaultReplica(region.getRegionInfo()) | |||
| .getRegionNameAsString(), | |||
| region.getRegionInfo().getRegionNameAsString(), counter.getAttemptTimes(), e); | |||
| try { | |||
There was a problem hiding this comment.
You did not get what I was saying I believe.
This is triggerFlushInPrimaryRegion() method and u can see a while loop within which the call happening. The while loop is to be terminated once this RS is set to be stopped/abort. When such happens, there might be many non daemon threads running within the server. Our logic in different places will interrupt these threads. so if the thread is sleeping or waiting it will get InterruptedException and allow the thread NOT to continue in running/waiting state. The logic should be checking the server status and allow to come out of loops etc.
But your change will make it such that even if the main thread interrupt this thread, it will continue to sleep for the specified time. That is totally against our intent.
There was a problem hiding this comment.
Ok got it. Yes this example makes sense. Are you saying that similar to this one, all the other places also need to handle Interruptions and there is no need to have uninterrupted sleep during client side retries?
There was a problem hiding this comment.
I did not check all places.. Most of the places where the RetryCounter is being used is in test I can see. But within RetryCounter we should not change. Tomorrow some other code path might use it too. IMO we can just keep the code as is. Let the calling part handle the InterruptedException the way they want. we can not generalise it. So just close this Jira Viraj
There was a problem hiding this comment.
Okk just saw that many places are in asynchronously getting executed in separate threads from main thread and if this is how interruptions were planned to be handled, it's fine. No need to make change.
…am#throttle should use uninterrupted sleep