bug: cannot use file-logger to log to stdout

[Alasiac]

Current Behavior

When using the file-logger plugin, it is possible to send the logging to stdout by defining /dev/stdout. This in order to use the docker output and relais this in kubernetes to a ELK stack. Though an error is thrown indicating a permission denied for the current user.

Expected Behavior

Be able to use /dev/stdout as a destination with the file-logger plugin.

Error Logs

failed to open file: /dev/stdout, error info: /dev/stdout: Permission denied while logging request

Steps to Reproduce

Run apisix docker with apisix.yaml:

routes:
  - uri: /*
    upstream_id: 1
    plugins:
      file-logger:
        path: /dev/stdout

upstreams:
  - id: 1
    nodes:
      "host.docker.internal:3000": 1
    type: roundrobin

plugins:
  - name: file-logger
#END

Environment

• APISIX Docker version: apache/apisix:2.13.0-alpine
• Host system: Macbook Pro Intel with docker desktop for mac

[spacewander]

Write log to non-regular file is not supported yet.
There is a plan to support it in the next release.

[spacewander]

We need to add specific logic around io.open to handle stdout/stderr, like io.stderr or io.stdout:
https://github.com/apache/apisix/blob/fc3a1c9c7403b4517dcbf760329cf9e4d2c5a5af/apisix/plugins/file-logger.lua

[Chever-John]

I think I can cover this issue.
Cause I have experience about this issue.
And I want to learn more about docker and APISIX.

The date I committed to is April 11.
If over, anyone can continue with this issue.

[Chever-John]

Hi, if possible, could you provide more dockerfile to reproduce your problem more accurately. @Alasiac Alasiac

The /dev/stdout is your macOS?

[Chever-John]

I am sorry that I am unable to reproduce this issue.
Can you provide a complete reproduction of the process?

@Alasiac

[Alasiac]

apisix-file-logger-problem.zip

A bit late, but as requested my usecase which results in
2022/06/22 13:55:09 [error] 48#48: *2514 [lua] file-logger.lua:117: write_file_data(): failed to open file: /dev/stdout, error info: /dev/stdout: Permission denied while logging request, client: 192.168.32.1,...

Nothing more than the file logger config as stated in the original issue in combination with the docker image of apisix.

[Chever-John]

@Alasiac Got it~
Thanks a lot for these files!!!

[e1ijah1]

Is this issue still available? If so, could you please assign it to me? I'm interested in trying it out.

[Chever-John]

@e1ijah1 Sure, Please @spacewander .

Best wish:P

[ro4i7]

Hello @spacewander @Alasiac

if this issue still open, please assign it to me and give feedback on below solution:

To solve this issue, we need to add specific logic around io.open to handle stdout and stderr like io.stderr or io.stdout. Currently, writing logs to non-regular files is not supported in the file-logger plugin. However, there is a plan to support it in the next release.

Here are the steps to fix this issue:

1. Modify the file-logger.lua plugin file to handle stdout and stderr:

local function open_file(self, conf)
  if not conf.path then
    return nil, "missing 'path'"
  end

  if conf.path == "/dev/stdout" then
    return io.stdout
  elseif conf.path == "/dev/stderr" then
    return io.stderr
  else
    local file, err = io.open(conf.path, "a+")
    if not file then
      return nil, "failed to open file: " .. conf.path .. ", error info: " .. tostring(err)
    end
    return file
  end
end

2. Build a custom image of Apisix with the modified file-logger.lua file included.

3. Use the custom image to run Apisix.

With these changes, the file-logger plugin should be able to write logs to stdout and stderr. This will allow us to send the logging to stdout by defining /dev/stdout and relay it in Kubernetes to an ELK stack.

[spacewander]

Have you tested the modification in file-logger.lua? The fix is like an early version of #8681, which has been proven is not worked. Since there is an ongoing PR for it, let's wait for the author's feedback.

[ro4i7]

Have you tested the modification in file-logger.lua? The fix is like an early version of #8681, which has been proven is not worked. Since there is an ongoing PR for it, let's wait for the author's feedback.

No, I haven't tested it yet. Just waiting for the author's feedback.

[dtotopus]

Hm, I'm a bit confused to be honest, because I just tried the following file-logger configuration prior seeing this thread and successfully logged the request { "include_resp_body": true, "path": "/dev/stdout" }

[flearc]

Is this issue still in progress? I noticed that the pr #8681 has been closed. @e1ijah1 @spacewander

[spacewander]

I have reviewed the PR #8681. The direction of this PR is correct as far as I know, but there are some issues in the latest review: #8681 (review)