Output to Log file changes the destination of stdout output plugin too

[aggarwalShivani]

Describe the bug

Requirement:
Fluentd runs as a kubernetes container
i.) Fluentd's own logs should be redirected to a log file
ii.) Fluentd is expected to read some other data from other plugins like in_http and emit that data onto stdout.

Issue:
To achieve (i.), -o command-line option is used as suggested at https://docs.fluentd.org/deployment/logging#by-command-line-option-1

$ fluentd -o /var/log/fluentd.log

To achieve (ii.), a match with stdout output plugin is added in the config.
For ex.

<source>
  @type http
  port 9880
  bind 0.0.0.0
  tag test
</source>

<match test>
    @type stdout
 </match>

If we configure fluentd process to output to a log-file, it also redirects the data (tagged test) intended for stdout, into the /var/log/fluentd.log file. It does not emit anything on stdou anymore.

Is this expected?

How to avoid it such that the log location and stdout location are separate?

To Reproduce

1. Configure fluentd config (as shared above)
2. Start fluentd with -o option
   fluentd -o /var/log/fluentd.log
3. Send data on the http input. For ex
   curl -X POST -d 'json={"foo":"bar"}' http://localhost:9880/test

Expected behavior

Fluentd own logs must go to logfile.
Records read from http must be present on stdout of fluentd

Your Environment

- Fluentd version: 1.16.9
- Package version:5.0.7
- Operating system: Rocky Linux 8.10
- Kernel version:6.6.87.2-microsoft-standard-WSL2

Your Configuration

<source>
  @type http
  port 9880
  bind 0.0.0.0
  tag test
</source>

<match test>
    @type stdout
 </match>

Your Error Log

No error log, there is no data emitted on stdout of fluentd container.

Additional context

No response

[Watson1978]

If we configure fluentd process to output to a log-file, it also redirects the data (tagged test) intended for stdout, into the /var/log/fluentd.log file. It does not emit anything on stdou anymore.

Is this expected?

It is expected.

What kind of use case is this issue?

[aggarwalShivani]

Basically, in the use-case, fluentd acts as a log-collector and processor for application logs.
It reads app logs from some source (like http), processes those logs (enriching some fields, changing format etc) and then emitting those logs onto its stdout (using match @stdout)

Now, fluentd's own logs also need to be enriched this way, so it requires that fluentd's own logs get redirected to a file, that fluentd itself can read, enrich and then emit onto its stdout.

[daipom]

Fluentd can handle its own logs by using @FLUENT_LOG:

• https://docs.fluentd.org/deployment/logging#capture-fluentd-logs

Does this resolve your issue?

[aggarwalShivani]

I'm already using @FLUENT_LOG in the configuration to process fluentd's own logs.
Point is -- when fluentd process starts, its logs would anyways go to some location -- stdout by default, or it could be a file if configured with -o file.log.

1. Fluentd process is started as /usr/bin/fluentd (default)

• Fluentd emits its logs to stdout
• Fluentd config has label @FLUENT_LOG to capture fluentd logs, process them and send them to a destination (stdout)

<label @FLUENT_LOG>
  <filter fluent.*>
    ## Process these logs, enrich them 
  </filter>
  <match fluent.*>
    @type stdout
  </match>
</label>

Result -> Fluentd's own logs get sent to stdout twice, once in original form and once in processed form.

2. Fluentd process is started as /usr/bin/fluentd -o <file.log>

• Fluentd own logs go to the file
• If there is any match section of @type stdout, that data also gets written to the same file.
  Result -> Nothing would show up on stdout.

[daipom]

I see! That was a blind spot.
To support that use case, one of the following two options would be required.

• Adding a new feature to disable the default logging. (Need to wait for a new version.)
• Creating an output plugin to output to STDOUT independently of the logger.

[daipom]

@aggarwalShivani
I have added use_logger option to out_stdout.

• out_stdout: support output to $stdout independently of logger #5036

With use_logger false, out_stdout outputs to STDOUT independently of Fluentd logger.
It will be released as Fluentd v1.19.0 soon.