[Enhancement]: Configurable options for Java GC logging

[bderrly]

Related problem

I would like to enable GC logging. However, it seems the only option is to toggle it on or off but not where to send the data (the default is to send to stdout). This make it difficult to separate the Kafka logs (also directed to stdout) from the GC logs in my log viewing system.

Suggested solution

I propose another configuration option to allow sending the GC log to a file (along with the other parameters such as log size and count of logs to keep). These options seem to be configurable by environment variables based upon my reading of Kafka's run script as well as Strimzi's run script.

Of course there is the issue of the container runtime and how the Kubernetes logging system works with only stdout and stderr streams. It seems the way, recommended by Kubernetes documentation, to manage this is with a sidecar container to read the log files within the Pod and write them to stdout.

Alternatives

None that I am aware of.

Additional context

No response

[scholzj]

This will be triaged. But I do not think we are planning to add any support for logging into a file currently. That would require having a way to maintain the files, manage and configure storage for them etc. So it would be a non-trivial task.

[bderrly]

I agree it would be non-trivial. Thanks for taking the time to triage this. I'm still relatively new to Kubernetes so it's possible that there is something that I am missing that would make it easier to separate the different log format streams such that they are not intermingled into stdout but I haven't found what that is yet. I look forward to seeing what you have to say on the issue.

[scholzj]

I don't think there is anything simple at the container level - at least I'm not aware of anything. You can of course do all kinds of post-processing once the logs are collected. You can for example use some pattern matching to separate the GC logs from the regular logs. I do not remember out of my head what the GC logs look like, but IMHO they had a distinctively different pattern.

[scholzj]

Triaged on 20.4.2023: There are already two existing issues covering two key parts of this issue:

• [Enhancement] Possibility to configure more containers in a Pod (sidecars, init containers) #2976 is about adding support for configuring sidecars (It is currently triaged and labeled as help-wanted / proposal-needed)
• Support mounting a volume to the pod #3693 is about adding support for adding additional volumes to the operands (It is currently triaged and labeled as help-wanted / proposal-needed)

These two would be a prerequisites to implement this. Until then, it does not seem to make sense to add any additional GC logging configuration since there would be no way to process it. So I guess it might make sense to close this issue for the time being?

---

It would be also good to understand why would be the information you would expect to get from the GC logs which would not be available for example in Prometheus metrics.

[scholzj]

Triaged on 4.5.2023: The author did not get back to us with the information about the use-case, what would nto be available from the Prometheus metrics etc. But the issues linked above should provide this feature if they were implemented. So this should be closed.

[piyushbhojgude]

Hi @scholzj , how do we enable Garbage collector logging for strimzi kafka connect?