Skip to content

Commit

Permalink
Rewrite debugging-eventlisteners.md for clarity and flow
Browse files Browse the repository at this point in the history
Rewrites debugging-eventlisteners.md for clarity and flow in preparation for creating a dedicated "Troubleshooting Triggers" page.
  • Loading branch information
tualeron authored and tekton-robot committed Mar 18, 2021
1 parent 02c3896 commit 5a59253
Showing 1 changed file with 38 additions and 36 deletions.
74 changes: 38 additions & 36 deletions docs/debugging-eventlisteners.md
Original file line number Diff line number Diff line change
@@ -1,43 +1,50 @@
<!--
---
linkTitle: "Debugging EventListeners"
linkTitle: "Troubleshooting"
---
-->
# EventListener logs
# Troubleshooting `EventListeners`

From the command-line, the easiest way to get the logs is:
This page describes the debugging methods you can use to diagnose and fix issues with Tekton Triggers.

## Gathering `EventListener` logs

You can gather `EventListener` logs using the Tekton `tkn` CLI tool or the Kubernetes `kubectl` CLI tool.

Use the following `tkn` command to gather `EventListener` logs:

```shell
$ tkn eventlistener logs
```
See [here](https://github.com/tektoncd/cli/blob/main/docs/cmd/tkn_eventlistener_logs.md) for more uses of the config.
See the `tkn` CLI tool [documentation page](https://github.com/tektoncd/cli/blob/main/docs/cmd/tkn_eventlistener_logs.md) for this config for more information.

Alternatively, you can get them with kubectl:
Use the following `kubectl` command to gather `EventListener` logs:

```shell
$ kubectl logs deploy/el-<insert name of eventlistener> -n <namespace>
```
You can get a list of EventListeners in a namespace with:

To get a list of `EventListeners` for a given namespace, use the following command:

```shell
$ kubectl get el -n <namespace>
NAME ADDRESS AVAILABLE REASON
test-event-listener http://el-test-event-listener.default.svc.cluster.local:8080 True MinimumReplicasAvailable
```

## Enable Debug Logging

Triggers creates a Kubernetes [ConfigMap](https://kubernetes.io/docs/concepts/configuration/configmap/) in the namespace that the EventListener
is created in called `config-logging-triggers`.
## Configuring debug logging for `EventListeners`

You can see this with:
By default, Tekton Triggers creates a Kubernetes [`ConfigMap`](https://kubernetes.io/docs/concepts/configuration/configmap/) named `config-logging-triggers`
in the namespace of the target `EventListener`. To view this `ConfigMap`, use the following command:

```shell
$ kubectl get cm -n <namespace>
NAME DATA AGE
config-logging-triggers 2 28m
```

Below is a typical `config-logging-listeners` `ConfigMap`:

```yaml
apiVersion: v1
kind: ConfigMap
Expand All @@ -52,58 +59,53 @@ data:
"","timeEncoder": "iso8601","durationEncoder": "","callerEncoder": ""}}'
```
### Enable debug level logging
The simplest way to enable additional debug logging is:
To enable debug-level logging in Tekton Triggers, use the following command:
```shell
$ kubectl patch cm config-logging-triggers -n <namespace> -p '{"data": {"loglevel.eventlistener": "debug"}}'
```
The EventListener should log out that it has changed the log level:
The `EventListener` responds with a confirmation of the new logging level. For example:

```json
{"level":"info","ts":"2021-02-10T08:42:10.950Z","logger":"eventlistener","caller":"logging/config.go:207","msg":"Updating logging level for eventlistener from debug to info.","knative.dev/controller":"eventlistener","logging.googleapis.com/labels":{},"logging.googleapis.com/sourceLocation":{"file":"knative.dev/pkg@v0.0.0-20210107022335-51c72e24c179/logging/config.go","line":"207","function":"knative.dev/pkg/logging.UpdateLevelFromConfigMap.func1"}}
```

When `debug` level is enabled, each HTTP request that the EventListener receives
will be logged with additional information:
From this point on, every HTTP request that the `EventListener` logs contains additional information. For example:

```json
{"level":"debug","ts":"2021-02-10T08:42:30.915Z","logger":"eventlistener","caller":"sink/sink.go:93","msg":"EventListener: demo-event-listener in Namespace: default handling event (EventID: 9x4mb) with path /testing, payload: {\"testing\": \"value\"} and header: map[Accept:[*/*] Content-Length:[20] Content-Type:[application/x-www-form-urlencoded] User-Agent:[curl/7.61.1] X-Auth:[testing]]","knative.dev/controller":"eventlistener","/triggers-eventid":"9x4mb","logging.googleapis.com/labels":{},"logging.googleapis.com/sourceLocation":{"file":"github.com/tektoncd/triggers/pkg/sink/sink.go","line":"93","function":"github.com/tektoncd/triggers/pkg/sink.Sink.HandleEvent"}}
```
**WARNING**: The headers are logged out verbatim, if your client is sending any sort of sensitive data in the headers, these will be logged out.

### Return to info level debugging
**WARNING**: The `EventListener` logs all payload headers verbatim. This includes any sensitive information the headers might contain.

You can restore it to the default logging level `info` with:
To disable debug-level logging, use the following command:

```shell
$ kubectl patch cm config-logging-triggers -n <namespace> -p '{"data": {"loglevel.eventlistener": "info"}}'
```

## Debugging JSONPath issues
This returns the logging level to `info`.

## Troubleshooting JSONPath issues

You may see the following message in your logs:

```json
A{"level":"error","ts":"2021-02-10T08:43:47.409Z","logger":"eventlistener","caller":"sink/sink.go:230","msg":"failed to ApplyEventValuesToParams: failed to replace JSONPath value for param message: $(body.message): message is not found","knative.dev/controller":"eventlistener","/triggers-eventid":"c8f88","/trigger":"demo-trigger","logging.googleapis.com/labels":{},"logging.googleapis.com/sourceLocation":{"file":"github.com/tektoncd/triggers/pkg/sink/sink.go","line":"230","function":"github.com/tektoncd/triggers/pkg/sink.Sink.processTrigger"},"stacktrace":"github.com/tektoncd/triggers/pkg/sink.Sink.processTrigger\n\tgithub.com/tektoncd/triggers/pkg/sink/sink.go:230\ngithub.com/tektoncd/triggers/pkg/sink.Sink.HandleEvent.func1\n\tgithub.com/tektoncd/triggers/pkg/sink/sink.go:125"}
```
This means that the JSON body received by the interceptor likely doesn't have
the structure you expect, you will probably need to capture the request somehow
and inspect it.

## Debugging 'no X-Hub-Signature set' and 'no X-GitLab-Token header set'
This means that the selected `Interceptor` is unable to parse the structure of the received JSON payload. To troubleshoot this, you must capture and inspect the payload for inconsistencies.

GitHub sends a header `X-Hub-Signature` when it's sending a hook that is
configured with a secret, and GitLab sends the `X-GitLab-Token` header.
## Troubleshooting signature and token errors

This error can occur in two different ways, if you set a secret in the GitHub or
GitLab interceptor, but don't configure the corresponding hook with a secret,
you will trigger these errors.
When sending a hook protected by a secret, GitHub includes an `X-Hub-Signature` object in the header, while GitLab includes an `X-GitLab-Token` object.
You may see `no X-Hub-Signature set` and/or `no X-GitLab-Token header set` errors in your logs in one of the following scenarios:

Alternatively this can occur when you have unsigned requests being routed
through an interceptor that requires them.
* If you specify a secret in your `Interceptor` but don't specify it in the hook.
* You are sending unsigned payloads to an `Interceptor` that expects signed payloads.

This can be a side-effect of your Trigger's interceptor stack, as all
interceptors are processed concurrently, so triggers that have a GitHub
interceptor will log this out, even though another trigger might successfully
process the request.
Note that depending on how you have configured your Tekton Triggers stack, these errors may not indicate an actual problem. For example, your stack may
include some `Interceptors` that expect signed payloads and some that expected unsigned payloads. Since Tekton Triggers processes all `Interceptors`
concurrently, `Interceptors` that expect signed payloads will log the above errors, while `Interceptors` that expect unsigned payloads will process
those payloads successfully.

0 comments on commit 5a59253

Please sign in to comment.