getLogBufferByteRemainsSize method

[efimmatytsin]

Hello @tkowalcz! We need this method for determining how much byte remains to send. Later I want to implement drain on stop functionality, because right now we lost some part of logs when server shutdown.

[tkowalcz]

Sounds reasonable. I have plans to implement it in core - ensure log buffer is drained when agent thread is shut down. I guess you need this to wait for the draining to be complete?

[efimmatytsin]

@tkowalcz Yes! Thats it. But also we need the opportunity to do drain forcily. Public method at any appended or LoggingSystem will be great! If you need some help, please send me a message

[tkowalcz]

Thank you for your offer. I have implemented drain on close incore and log4j side complete with tests, but decided to do some refactoring of agent code. Should be done sometime next week.

[tkowalcz]

Hi @efimmatytsin I was wondering if you are still using this project and need any further enhancements?

[efimmatytsin]

Hello @tkowalcz ! Yes it really helpful project! Thank you very much! Actually, yes! We want some enhancement like sort batch by timestamp. Because loki didn't do that by their side.

[tkowalcz]

I'm glad to hear that @efimmatytsin. Regarding the dreaded out of order logs, I did not implement it for reasons:

1. Seems like a lot of work ;)
2. I think that there can be instances where unrelated agents (Tjahzi or promtail or anything else) write to same stream and there is no way of preventing this error.
3. Loki one day will lift that restriction.

So patience seems to have paid off and Loki will fix this on its side. See grafana/loki#4330 and https://grafana.com/blog/2021/09/16/avoid-dropped-logs-due-to-out-of-order-timestamps-with-a-new-loki-feature/

Cheers,
Tomasz