Is it possible to configure logstash stop parsing logs under certain conditions?

[jordansissel]

(This issue was originally filed by @alessiodelchiaro at elastic/logstash#2858)

---

Hi Guys,

Is there any chance to allow logstash stop parsing a specific log (e.g., when it reaches EOF)?

My application might generate GB of logs so I would like to remove them once those are parsed.

If it is not possible, it would be great to have this feature in a future version.

Thanks,

Alessio

[dev-head]

+1 I really want logstash to implement some flavor of post file processing.

It would be a significant improvement to have an option to stop watching a file after a configurable number of seconds since last event. That way a file input watcher could be expired; which could then make it safe for logstash to allow some post processing to the file such as: execute a script, move the file, truncate the file and or delete the file.

Right now I am relying on a fragile process that tries to remove old log files; because there's no way to guarantee all events have been processed and to have an effective cleanup process.

[jordansissel]

This is a common request. I am in favor of the approach, but I'm not sure how to expose it. We'll need a design first, I think, that covers the two file modes ("read file to EOF once, then do something" vs "read file forever, but when it's idle for 24 hours, then do something")