fluent bit: add identifier to help query logs in output (#2195)

* Add controller pod name to logs Signed-off-by: Sanya Kochhar <kochhars@microsoft.com>
openservicemesh · Jan 11, 2021 · 78e26d0 · 78e26d0
1 parent df88f4a
commit 78e26d0
Show file tree

Hide file tree

Showing 3 changed files with 25 additions and 5 deletions.
diff --git a/charts/osm/templates/fluentbit-configmap.yaml b/charts/osm/templates/fluentbit-configmap.yaml
@@ -34,11 +34,12 @@ data:
       Name           grep
       Match          kube.*
       Regex          keep true
-    # Removes extra "keep: true" key/value pair once matching is complete 
+    # Removes extra "keep: true" key/value pair once matching is complete; adds controller pod name value to help users query logs in output
     [FILTER]
       Name           modify
       Match          kube.*
       Remove         keep
+      Add            controller_pod_name ${CONTROLLER_POD_NAME}
     [OUTPUT]
       Name    {{ .Values.OpenServiceMesh.fluentBit.outputPlugin }}
       Match   *

diff --git a/charts/osm/templates/osm-deployment.yaml b/charts/osm/templates/osm-deployment.yaml
@@ -79,11 +79,24 @@ spec:
         - name: {{ .Values.OpenServiceMesh.fluentBit.name }}
           image: {{ .Values.OpenServiceMesh.fluentBit.registry }}/fluent-bit:{{ .Values.OpenServiceMesh.fluentBit.tag }}
           imagePullPolicy: {{ .Values.OpenServiceMesh.fluentBit.pullPolicy }}
-        {{- if .Values.OpenServiceMesh.fluentBit.enableProxySupport }}
-          envFrom:
-            - secretRef:
+          env:
+          {{- if .Values.OpenServiceMesh.fluentBit.enableProxySupport }}
+          - name: HTTP_PROXY
+            valueFrom:
+              secretKeyRef:
+                name: proxy-config
+                key: HTTP_PROXY
+          - name: HTTPS_PROXY
+            valueFrom:
+              secretKeyRef:
                 name: proxy-config
-        {{- end }}
+                key: HTTPS_PROXY
+          {{- end }}
+          - name: CONTROLLER_POD_NAME
+            valueFrom:
+              fieldRef:
+                apiVersion: v1
+                fieldPath: metadata.name
           volumeMounts:
           - name: config
             mountPath: /fluent-bit/etc

diff --git a/docs/patterns/observability/logs.md b/docs/patterns/observability/logs.md
@@ -23,6 +23,8 @@ To customize log forwarding to your output, follow these steps and then reinstal
 
 4. Once you have updated the Fluent Bit configmap, you can deploy the sidecar during OSM installation using the `--enable-fluentbit` flag. You should now be able to interact with error logs in the output of your choice as they get generated.
 
+5. The `controller_pod_name` key/value pair has been added to the logs to help you query logs in your output by refining results on pod name (see example usage below).
+
 ### Example: Using Fluent Bit to send logs to Azure Monitor
 Fluent Bit has an Azure output plugin that can be used to send logs to an Azure Log Analytics workspace as follows:
 1. [Create a Log Analytics workspace](https://docs.microsoft.com/en-us/azure/azure-monitor/learn/quick-create-workspace)
@@ -36,6 +38,10 @@ Fluent Bit has an Azure output plugin that can be used to send logs to an Azure
     fluentbit_CL
     | order by TimeGenerated desc
     ```
+5. Refine your log results on a specific deployment of the OSM controller pod:
+    ```
+    | where controller_pod_name_s == "<desired osm controller pod name>"
+    ```
 
 Once logs have been sent to Log Analytics, they can also be consumed by Application Insights as follows:
 1. [Create a Workspace-based Application Insights instance](https://docs.microsoft.com/en-us/azure/azure-monitor/app/create-workspace-resource).