Increase buffer size for fluent bit - lines are too long, skipping file

[msmith99]

Error message -

[input:tail:tail.0] file=/server.log requires a larger buffer size, lines are too long. Skipping file."

Root cause, is that the buffer size configured for fluent bit is too small, either this needs to be increased, or made configurable -

[INPUT]
Name tail
Path /home/logs/server/*.log
Db /tmp/ng.db
Db.sync Full
Tag nginx-log
Buffer_Chunk_Size 320KB
Buffer_Max_Size 520KB

[msmith99]

This can be related to #30, which is an enhancement to expose the fluent bit config, which could them be used to increase the buffer size.

[drozinpax8]

I'm getting the same error with application running on tomcat:

file=/usr/share/tomcat8/logs/app.log requires a larger buffer size, lines are too long. Skipping file.

Do we have a timeline to add this configuration?

[fgrelaud]

Hi,

do we have any news about this issue ?
we have same problem with nextcloud log.

file=/var/log/nextcloud/nextcloud.log requires a larger buffer size, lines are too long. Skipping file.

Regards,

[kasia-kujawa]

Currently the only solution to change configuration for sidecar image is to built new sidecar image with custom configuration because configuration is copied into image, see these lines.

Existing configuration is here: https://github.com/SumoLogic/tailing-sidecar/tree/main/sidecar/conf
Building instruction for sidecar image can be found here: https://github.com/SumoLogic/tailing-sidecar/tree/7fd074684531f6816536f5f5ac29f721dacbc26e/sidecar#build-and-push-tailing-sidecar-to-docker-registry

and then to use custom sidecar image it is necessary to change helm chart configuration in values.yaml by modifying sidecar image, please see these lines.

[kasia-kujawa]

I'm closing this issue, if you still observe the issue, please reopen it ;)