access-log: Add JSON support #1350
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Raphael Taylor-Davies <r.taylordavies@googlemail.com>
Signed-off-by: Raphael Taylor-Davies <r.taylordavies@googlemail.com>
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Signed-off-by: Raphael Taylor-Davies <r.taylordavies@googlemail.com>
Signed-off-by: Raphael Taylor-Davies <r.taylordavies@googlemail.com>
Signed-off-by: Raphael Taylor-Davies <r.taylordavies@googlemail.com>
Signed-off-by: Raphael Taylor-Davies <r.taylordavies@googlemail.com>
Signed-off-by: Raphael Taylor-Davies <r.taylordavies@googlemail.com>
The `Settings::build` method is fairly large and complex. This change splits into a few smaller units and reduces boilerplate--especially around reading the settings from the environment. This change is being made in service of #601 to help avoid making the build method even more complicated.
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
may be a tracing bug! Signed-off-by: Eliza Weisman <eliza@buoyant.io>
this fixes the issue with adding a layer filter to the `fmt` layer; i forgot the access log layer was relying on the `fmt` layer to format the span fields. anyway this fixes that. Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Co-authored-by: Oliver Gould <ver@buoyant.io>
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Depends on #1319. This branch adds the option to configure the access logging layer to output JSON rather than the Apache Common access log format. Currently, this is configured based on whether or not the file specified in the `LINKERD2_PROXY_ACCESS_LOG` environment variable ends in `.json`. We could also add a separate environment variable for the access log format, but I thought this approach made more sense... A bit of sample output: ```json {"client.addr":"127.0.0.1:51764","client.id":"-","host":"transparency.test.svc.cluster.local","method":"GET","processing_ns":"836068","request_bytes":"","response_bytes":"0","status":200,"timestamp":"2021-11-01T21:54:28.844393794Z","total_ns":"4225011","trace_id":"","uri":"http://transparency.test.svc.cluster.local/","user_agent":"","version":"HTTP/2.0"} {"client.addr":"127.0.0.1:46812","client.id":"-","host":"transparency.test.svc.cluster.local","method":"GET","processing_ns":"917210","request_bytes":"","response_bytes":"8","status":200,"timestamp":"2021-11-01T21:54:28.843909802Z","total_ns":"4566057","trace_id":"","uri":"http://transparency.test.svc.cluster.local/","user_agent":"","version":"HTTP/2.0"} {"client.addr":"127.0.0.1:34510","client.id":"-","host":"transparency.test.svc.cluster.local","method":"POST","processing_ns":"617010","request_bytes":"","response_bytes":"0","status":200,"timestamp":"2021-11-01T21:54:28.852085211Z","total_ns":"1428744","trace_id":"","uri":"http://transparency.test.svc.cluster.local/","user_agent":"","version":"HTTP/2.0"} {"client.addr":"127.0.0.1:34510","client.id":"-","host":"transparency.test.svc.cluster.local","method":"PUT","processing_ns":"445462","request_bytes":"","response_bytes":"0","status":200,"timestamp":"2021-11-01T21:54:28.855927578Z","total_ns":"1094081","trace_id":"","uri":"http://transparency.test.svc.cluster.local/","user_agent":"","version":"HTTP/2.0"} {"client.addr":"127.0.0.1:34510","client.id":"-","host":"transparency.test.svc.cluster.local","method":"DELETE","processing_ns":"389387","request_bytes":"","response_bytes":"0","status":200,"timestamp":"2021-11-01T21:54:28.859080459Z","total_ns":"944251","trace_id":"","uri":"http://transparency.test.svc.cluster.local/","user_agent":"","version":"HTTP/2.0"} {"client.addr":"127.0.0.1:34510","client.id":"-","host":"transparency.test.svc.cluster.local","method":"HEAD","processing_ns":"404264","request_bytes":"","response_bytes":"0","status":200,"timestamp":"2021-11-01T21:54:28.862128314Z","total_ns":"978064","trace_id":"","uri":"http://transparency.test.svc.cluster.local/","user_agent":"","version":"HTTP/2.0"} {"client.addr":"127.0.0.1:34510","client.id":"-","host":"transparency.test.svc.cluster.local","method":"PATCH","processing_ns":"404874","request_bytes":"","response_bytes":"0","status":200,"timestamp":"2021-11-01T21:54:28.865139581Z","total_ns":"996539","trace_id":"","uri":"http://transparency.test.svc.cluster.local/","user_agent":"","version":"HTTP/2.0"} {"client.addr":"127.0.0.1:34510","client.id":"-","host":"transparency.test.svc.cluster.local","method":"GET","processing_ns":"982353","request_bytes":"","response_bytes":"0","status":200,"timestamp":"2021-11-01T21:54:28.844075551Z","total_ns":"5095004","trace_id":"","uri":"http://transparency.test.svc.cluster.local/","user_agent":"","version":"HTTP/2.0"} ```
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
olix0r
reviewed
Jan 21, 2022
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
olix0r
reviewed
Jan 21, 2022
olix0r
reviewed
Jan 21, 2022
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Co-authored-by: Oliver Gould <ver@buoyant.io>
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
olix0r
approved these changes
Jan 21, 2022
olix0r
added a commit
to linkerd/linkerd2
that referenced
this pull request
Jan 21, 2022
This release adds a new configuration variable, `LINKERD2_PROXY_ACCESS_LOG` that configures whether per-request access logs are emitted from inbound HTTP proxies. This variable may be set to `apache` for Apache Common Log output or `json` for JSON formatted logs. Access logs are emitted to stderr and the proxy's diagnostics logs are emitted to stdout. --- * build(deps): bump h2 from 0.3.9 to 0.3.10 (linkerd/linkerd2-proxy#1430) * Update Rust to v1.56.1 (linkerd/linkerd2-proxy#1431) * build(deps): bump indexmap from 1.7.0 to 1.8.0 (linkerd/linkerd2-proxy#1433) * Add a devcontainer config (linkerd/linkerd2-proxy#1432) * build(deps): bump serde_json from 1.0.74 to 1.0.75 (linkerd/linkerd2-proxy#1436) * build(deps): bump tracing-subscriber from 0.3.5 to 0.3.6 (linkerd/linkerd2-proxy#1435) * test: remove `env::set_var` call in test logging initialization (linkerd/linkerd2-proxy#1437) * trace: Apache Common Log Format access logging (linkerd/linkerd2-proxy#1319) * build(deps): bump libc from 0.2.112 to 0.2.113 (linkerd/linkerd2-proxy#1438) * build(deps): bump socket2 from 0.4.2 to 0.4.3 (linkerd/linkerd2-proxy#1439) * access-log: Add JSON support (linkerd/linkerd2-proxy#1350)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Depends on #1319.
This branch adds the option to configure the access logging layer to
output JSON rather than the Apache Common access log format.
Currently, this is configured based on whether or not the file specified
in the
LINKERD2_PROXY_ACCESS_LOGenvironment variable ends in.json.We could also add a separate environment variable for the access log
format, but I thought this approach made more sense...
A bit of sample output:
{"client.addr":"127.0.0.1:51764","client.id":"-","host":"transparency.test.svc.cluster.local","method":"GET","processing_ns":"836068","request_bytes":"","response_bytes":"0","status":200,"timestamp":"2021-11-01T21:54:28.844393794Z","total_ns":"4225011","trace_id":"","uri":"http://transparency.test.svc.cluster.local/","user_agent":"","version":"HTTP/2.0"} {"client.addr":"127.0.0.1:46812","client.id":"-","host":"transparency.test.svc.cluster.local","method":"GET","processing_ns":"917210","request_bytes":"","response_bytes":"8","status":200,"timestamp":"2021-11-01T21:54:28.843909802Z","total_ns":"4566057","trace_id":"","uri":"http://transparency.test.svc.cluster.local/","user_agent":"","version":"HTTP/2.0"} {"client.addr":"127.0.0.1:34510","client.id":"-","host":"transparency.test.svc.cluster.local","method":"POST","processing_ns":"617010","request_bytes":"","response_bytes":"0","status":200,"timestamp":"2021-11-01T21:54:28.852085211Z","total_ns":"1428744","trace_id":"","uri":"http://transparency.test.svc.cluster.local/","user_agent":"","version":"HTTP/2.0"} {"client.addr":"127.0.0.1:34510","client.id":"-","host":"transparency.test.svc.cluster.local","method":"PUT","processing_ns":"445462","request_bytes":"","response_bytes":"0","status":200,"timestamp":"2021-11-01T21:54:28.855927578Z","total_ns":"1094081","trace_id":"","uri":"http://transparency.test.svc.cluster.local/","user_agent":"","version":"HTTP/2.0"} {"client.addr":"127.0.0.1:34510","client.id":"-","host":"transparency.test.svc.cluster.local","method":"DELETE","processing_ns":"389387","request_bytes":"","response_bytes":"0","status":200,"timestamp":"2021-11-01T21:54:28.859080459Z","total_ns":"944251","trace_id":"","uri":"http://transparency.test.svc.cluster.local/","user_agent":"","version":"HTTP/2.0"} {"client.addr":"127.0.0.1:34510","client.id":"-","host":"transparency.test.svc.cluster.local","method":"HEAD","processing_ns":"404264","request_bytes":"","response_bytes":"0","status":200,"timestamp":"2021-11-01T21:54:28.862128314Z","total_ns":"978064","trace_id":"","uri":"http://transparency.test.svc.cluster.local/","user_agent":"","version":"HTTP/2.0"} {"client.addr":"127.0.0.1:34510","client.id":"-","host":"transparency.test.svc.cluster.local","method":"PATCH","processing_ns":"404874","request_bytes":"","response_bytes":"0","status":200,"timestamp":"2021-11-01T21:54:28.865139581Z","total_ns":"996539","trace_id":"","uri":"http://transparency.test.svc.cluster.local/","user_agent":"","version":"HTTP/2.0"} {"client.addr":"127.0.0.1:34510","client.id":"-","host":"transparency.test.svc.cluster.local","method":"GET","processing_ns":"982353","request_bytes":"","response_bytes":"0","status":200,"timestamp":"2021-11-01T21:54:28.844075551Z","total_ns":"5095004","trace_id":"","uri":"http://transparency.test.svc.cluster.local/","user_agent":"","version":"HTTP/2.0"}