Async append blocking with timeout #565
Conversation
brenuart
force-pushed
the
gh559-async-block
branch
3 times, most recently
from
f15373d
to
64901f5
Compare
brenuart
force-pushed
the
gh559-async-block
branch
from
d28f43d
to
184a265
Compare
Calling Disruptor#shutdown(timeout) while the buffer is not empty causes the disruptor to wait in a busy-loop consommuing a lot of CPU. Instead, wait during the grace period before asking the disruptor to shutdown immediately. Related issue: logfellow#566
Block/drop behaviour is entirely controlled by the value of the “appendTimeout” property: - `-1` to disable timeout and wait until enough space becomes available - `0` for no timeout at all and drop the event immediately when the buffer is full - `> 0` to retry during the specified amount of time
brenuart
force-pushed
the
gh559-async-block
branch
from
184a265
to
db89593
Compare
|
(rebased on main) |
src/main/java/net/logstash/logback/appender/AsyncDisruptorAppender.java
Outdated
Show resolved
Hide resolved
I actually meant #566. |
Got it. Thanks. |
|
The PR looks go enough to me now. |
…e the same section
|
Looks good! I'll squash and merge after you resolve the existing conflicts. |
brenuart
changed the title
Please have a look at this first draft implementation for #559.
In short, the appender supports two distinct operation modes (set via
asyncMode):DROP(the default): the appender drops events when the ring buffer is full. This is the legacy behaviour.BLOCK: the appending thread is blocked when the buffer is full until some free space becomes available. An optional timeout can be specified using theretryTimeoutconfiguration property (disabled by default).The RingBuffer does not allow me to "publish with a timeout" - it offers either a "try publish" or a "publish". The later blocks indefinitely until there is enough space in the buffer to accept the event. Even worst, waiting threads are not unblocked when the disruptor is shutdown.
Instead, I decided to tryPublish and sleep before retrying until the optional timeout expires. Sleeping is done using
LockSupport.parkNanos()just like what the Disruptor does too. Retries are scheduled every 100ms which may appear to be too frequent (CPU?). The problem is there is no way I'm aware of that tells me when there is room in the ring buffer. May be I can hook into the EvenHandler and awake any waiting threads when an event has been processed... This means a queue of waiting threads and potential synchronisation issues and/or performance impacts...Instead of
asyncModeandretryTimeout(feel free to comment on the name of these properties) it may be more convenient to collapse them into a singleappendTimeoutconfiguration property whose semantic would be:-1: timeout is disabled, means block until space is available (aka theBLOCKmode)0: times out immediately (aka theDROPmode) - this would be the default behaviour>0: times out after the given timeWhen the appender times out, it drops the event and logs a warning as it does already.
@philsttr Feedback?