Merge pull request #11 from IBM/suid-support

Suid support

.gitignore

@@ -1,2 +1,4 @@
 /target/
-**/output/**
+**/output/**
+coreos.Dockerfile
+*.vscode*

CONTRIBUTING.md

@@ -0,0 +1,79 @@
+# Contributing
+
+Our project welcomes external contributions. If you have an itch, please feel
+free to scratch it.
+
+It should also be noted that **[core-dump-handler](https://github.com/IBM/core-dump-handler/) is an [_OPEN Open Source Projects_](https://openopensource.org/).**
+
+Individuals making significant and valuable contributions are given commit-access to a project to contribute as they see fit. A project is more like an open wiki than a standard guarded open source project.
+
+To contribute minor code or documentation, please submit a [pull request](https://github.com/ibm/core-dump-handler/pulls).
+
+A good way to familiarize yourself with the codebase and contribution process is
+to look for and tackle low-hanging fruit in the [issue tracker](https://github.com/IBM/core-dump-handler/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22).
+Before embarking on a more ambitious contribution, please quickly [get in touch](#communication) with us.
+
+**Note: We appreciate your effort, and want to avoid a situation where a contribution
+requires extensive rework (by you or by us), sits in backlog for a long time, or
+cannot be accepted at all!**
+
+
+## Releases
+
+Declaring formal releases remains the prerogative of the project maintainer(s).
+
+### Proposing new features
+
+#### First time contributors 
+
+If you would like to implement a new feature, please [raise an issue](https://github.com/ibm/core-dump-handler/issues)
+before sending a pull request so the feature can be discussed. This is to avoid
+you wasting your valuable time working on a feature that the project developers
+are not interested in accepting into the code base.
+
+#### On-boarded collaborators
+
+1. **No `--force` pushes** or modifying the Git history in any way.
+1. **Non-main branches** ought to be used for ongoing work.
+1. **External API changes and significant modifications** ought to be subject to an **internal pull-request** to solicit feedback from other contributors.
+1. Internal pull-requests to solicit feedback are *encouraged* for any other non-trivial contribution but left to the discretion of the contributor.
+
+### Fixing bugs
+
+If you would like to fix a bug, please feel free to open a [PR directly for a small change](https://github.com/ibm/core-dump-handler/pulls).
+If you think the fix will be high impact then consider [opening an issue](https://github.com/ibm/repo-template/issues) before sending a
+pull request so it can be tracked.
+
+### Merge approval
+
+For first time PRs the project maintainers use LGTM (Looks Good To Me) in comments on the code
+review to indicate acceptance.
+
+For a list of the maintainers, see the [CONTRIBUTORS.md](CONTRIBUTORS.md) page.
+
+
+# Legal conditions
+
+- Any contributions (code, information etc) submitted will be subject to the same [license](LICENSE) as the rest of the code.
+No new restrictions/conditions are permitted.
+- As a contributor, you MUST have the legal right to grant permission for your contribution to be used under these conditions.
+
+## Communication
+Please use the [issue list] to keep communication transparent (https://github.com/ibm/repo-template/issues)
+
+## Setup
+The quickest way to get setup is to use a [free cluster](https://cloud.ibm.com/docs/containers?topic=containers-getting-started#clusters_gs) on IBM Cloud so you can test your work. 
+
+Instructions on how to install are available in the main [README.md](https://github.com/IBM/core-dump-handler#installing-the-chart)
+
+## Testing
+Currently there are unit tests for the agent and composer projects.
+The tests need to be ran as `root` on a Linux machine and will modify system settings.
+PR's that modify the codebase will be expected to run against a cluster before being accepted.
+
+## Coding style guidelines
+Code contributions should be PR'd with `cargo fmt` ran 
+
+**[core-dump-handler](https://github.com/IBM/core-dump-handler/) is an [_OPEN Open Source Projects_](https://openopensource.org/).**
+
+Individuals making significant and valuable contributions are given commit-access to a project to contribute as they see fit. A project is more like an open wiki than a standard guarded open source project.

CONTRIBUTORS.md

@@ -0,0 +1,7 @@
+# Contributors
+
+Maintainers 
+
+| Name                         | GitHub                                                           | Social                                                        |
+| :--------------------------- | :--------------------------------------------------------------- | :------------------------------------------------------------ |
+| **Anton Whalley**         | [**@No9**](https://github.com/No9)           | [**@dhigit9**](https://twitter.com/dhigit9)    |

README.md

@@ -30,8 +30,8 @@ As the agent runs in privileged mode the following command is needed on OpenShif
 ```
 oc adm policy add-scc-to-user privileged -z core-dump-admin -n observe
 ```
-Some OpenShift services run on RHEL7 if that's the case then add the folowing option to the helm command or update the values.yaml.
-This will be apparent if you see errors relating to glibc in the output.log in the host directory core folder which can be accessed from the agent pod at `/core-dump-handler/core`
+Some OpenShift services such asOpenShift on IBM Cloud run on RHEL7 if that's the case then add the folowing option to the helm command or update the values.yaml.
+This will be apparent if you see errors relating to glibc in the composer.log in the install folder of the agent. [See Troubleshooting below](#troubleshooting)
 ```
 --set daemonset.vendor=rhel7
 ```
@@ -69,16 +69,16 @@ This is a matrix of confirmed test targets. Please PR environments that are also
     <td>Microsoft</td><td>AKS</td><td>1.19</td><td>Yes</td><td>Yes</td><td></td>
 </tr>
 <tr>
-    <td>Microsoft</td><td>ARO</td><td>?</td><td>No</td><td>Unknown</td><td></td>
+    <td>Microsoft</td><td>ARO</td><td>4.6</td><td>Yes</td><td>No</td><td>Runs on CoreOS and building building compatable binaries seems to be the next step</td>
 </tr>
 <tr>
-    <td>AWS</td><td>EKS</td><td>1.21</td><td>Yes</td><td>No crictl client in the default AMI means that the metadata won't be captured</td><td></td>
+    <td>AWS</td><td>EKS</td><td>1.21</td><td>Yes</td><td>Yes*</td><td>crictl client in the default AMI means that the metadata won't be captured</td>
 </tr>
 <tr>
-    <td>AWS</td><td>ROSA</td><td>?</td><td>No</td><td>Unknown</td><td></td>
+    <td>AWS</td><td>ROSA</td><td>4.6</td><td>Yes</td><td>No</td><td>Runs on CoreOS and building building compatable binaries seems to be the next step</td>
 </tr>
 <tr>
-    <td>Google</td><td>GKE</td><td>1.19</td><td>Yes</td><td>Possible</td><td>Default HostPath Fails A <a href="https://kubernetes.io/docs/concepts/storage/volumes/#local">local PV</a> needs to be defined</td>
+    <td>Google</td><td>GKE</td><td>1.19</td><td>Yes</td><td>Possible</td><td>Default HostPath Fails A <a href="https://kubernetes.io/docs/concepts/storage/volumes/#local">local PV</a> needs to be defined. Please [see issue 5](https://github.com/IBM/core-dump-handler/issues/5) for updates</td>
 </tr>
 </table>
 
@@ -176,7 +176,7 @@ helm delete coredump-handler -n observe
 
 ## Build and Deploy a Custom Version
 
-[![Docker Repository on Quay](https://quay.io/repository/number9/core-dump-handler/status "Docker Repository on Quay")](https://quay.io/repository/number9/core-dump-handler)
+[![Docker Repository on Quay](https://quay.io/repository/icdh/core-dump-handler/status "Docker Repository on Quay")](https://quay.io/repository/icdh/core-dump-handler)
 
 The services are written in Rust using [rustup](https://rustup.rs/).
 
@@ -190,4 +190,41 @@ The services are written in Rust using [rustup](https://rustup.rs/).
 image:
   repository: YOUR_TAG_NAME 
 ```
-or run the helm install command with the `--set image.repository=YOUR_TAG_NAME`.
+or run the helm install command with the `--set image.repository=YOUR_TAG_NAME`.
+
+## Troubleshooting 
+
+The first place to look for issues is in the agent console.
+A successful install should look like this
+
+```
+
+[2021-09-08T22:28:43Z INFO core_dump_agent] Setting host location to: /var/mnt/core-dump-handler
+[2021-09-08T22:28:43Z INFO core_dump_agent] Current Directory for setup is /app
+[2021-09-08T22:28:43Z INFO core_dump_agent] Copying the composer from ./vendor/default/cdc to /var/mnt/core-dump-handler/cdc
+[2021-09-08T22:28:43Z INFO core_dump_agent] Starting sysctl for kernel.core_pattern /var/mnt/core-dump-handler/core_pattern.bak
+[2021-09-08T22:28:43Z INFO core_dump_agent] Created Backup of /var/mnt/core-dump-handler/core_pattern.bak
+[2021-09-08T22:28:43Z INFO core_dump_agent] Starting sysctl for kernel.core_pipe_limit /var/mnt/core-dump-handler/core_pipe_limit.bak
+[2021-09-08T22:28:43Z INFO core_dump_agent] Created Backup of /var/mnt/core-dump-handler/core_pipe_limit.bak
+[2021-09-08T22:28:43Z INFO core_dump_agent] Starting sysctl for fs.suid_dumpable /var/mnt/core-dump-handler/suid_dumpable.bak
+[2021-09-08T22:28:43Z INFO core_dump_agent] Created Backup of /var/mnt/core-dump-handler/suid_dumpable.bak
+[2021-09-08T22:28:43Z INFO core_dump_agent] Created sysctl of kernel.core_pattern=|/var/mnt/core-dump-handler/cdc -c=%c -e=%e -p=%p -s=%s -t=%t -d=/var/mnt/core-dump-handler/core -h=%h -E=%E
+kernel.core_pattern = |/var/mnt/core-dump-handler/cdc -c=%c -e=%e -p=%p -s=%s -t=%t -d=/var/mnt/core-dump-handler/core -h=%h -E=%E
+kernel.core_pipe_limit = 128
+[2021-09-08T22:28:43Z INFO core_dump_agent] Created sysctl of kernel.core_pipe_limit=128
+fs.suid_dumpable = 2
+[2021-09-08T22:28:43Z INFO core_dump_agent] Created sysctl of fs.suid_dumpable=2
+[2021-09-08T22:28:43Z INFO core_dump_agent] Creating /var/mnt/core-dump-handler/.env file with LOG_LEVEL=info
+[2021-09-08T22:28:43Z INFO core_dump_agent] Executing Agent with location : /var/mnt/core-dump-handler/core
+[2021-09-08T22:28:43Z INFO core_dump_agent] Dir Content []
+```
+
+If the agent is running successfully then there may be a problem with the composer configuration.
+To check the logs for the composer open a shell into the agent and cat the composer.log to see if there are any error messages.
+
+```
+cat /var/mnt/core-dump-handler/composer.log
+```
+
+If there are no errors then you should change the default log from `error` to `debug` in the values.yaml and redeploy the chart.
+Create a core dump again and `/var/mnt/core-dump-handler/composer.log` should contain specific detail on each upload.

charts/Chart.yaml

@@ -6,6 +6,6 @@ type: application
 
 version: 1.0.0
 
-appVersion: 2.0.0
+appVersion: 2.1.0
 
 icon: https://raw.githubusercontent.com/No9/ibm-core-dump-handler/master/assets/handle-with-care-svgrepo-com.svg

charts/templates/daemonset.yaml

@@ -21,8 +21,12 @@ spec:
           mountPath:  {{ .Values.daemonset.hostDirectory }}
           mountPropagation: Bidirectional
         env:
+          - name: COMP_LOG_LEVEL
+            value: {{ .Values.daemonset.composerLogLevel }}
           - name: HOST_DIR
             value: {{ .Values.daemonset.hostDirectory }}
+          - name: SUID_DUMPABLE
+            value: {{ .Values.daemonset.suidDumpable | quote }}
           - name: S3_ACCESS_KEY
             value: {{ .Values.daemonset.s3AccessKey }}
           - name: S3_SECRET
@@ -33,6 +37,8 @@ spec:
             value: {{ .Values.daemonset.s3Region }}
           - name: VENDOR
             value: {{ .Values.daemonset.vendor }}
+          - name: INTERVAL
+            value: {{ .Values.daemonset.interval | quote }}
         command: ["/app/core-dump-agent"]
         lifecycle:
           preStop:

charts/values.yaml

@@ -1,7 +1,7 @@
 replicaCount: 1
 
 image:
-  repository: quay.io/number9/core-dump-handler:v2.0.11
+  repository: quay.io/icdh/core-dump-handler@sha256:b8f5c392640c9d228b0e32c5889b29c37bc330adace0c2f2d8464d6664e644ef
   pullPolicy: IfNotPresent
 
 imagePullSecrets: []
@@ -15,11 +15,14 @@ daemonset:
     name: "core-dump-handler"
     label: "core-dump-ds"
     hostDirectory: "/var/mnt/core-dump-handler"
+    composerLogLevel: "Warn"
+    suidDumpable: 2
     s3AccessKey : XXX
     s3Secret : XXX
     s3BucketName : XXX
     s3Region : XXX
     vendor: default
+    interval: 60000
 
 serviceAccount:
   create: true

core-dump-agent/Cargo.toml

@@ -9,4 +9,5 @@ edition = "2018"
 [dependencies]
 env_logger = "0.8.3"
 log = "0.4.14"
-rust-s3 = { version = "0.26.0"}
+rust-s3 = "0.26.0"
+advisory-lock = "0.3.0"