containers · rhatdan · Jul 2, 2024 · Jul 2, 2024
@@ -1,7 +1,7 @@
 # QM is a containerized environment for running Functional Safety qm (Quality Management) software
 
 The main purpose of this package is allow users to setup an environment which
-prevents applications and container tools from interfering with other processes 
+prevents applications and container tools from interfering with other processes
 on the system. For example ASIL (Automotive Safety Integrity Level) environments.
 
 The QM environment uses containerization tools like cgroups, namespaces, and
@@ -32,12 +32,13 @@ each other.
 For now all of the control processes in the qm other then containers will run
 with the same qm_t type.
 
-Still would like to discuss about a specific selinux prevision?  
+Still would like to discuss about a specific selinux prevision?
 Please open an [QM issue](https://github.com/containers/qm/issues) with the output of selinux error from a recent operation related to QM. The output of the following commands are appreciated for understanding the root cause.
-```
-ausearch -m avc -ts recent | audit2why  
-journalctl -t setroubleshoot  
-sealert -a /var/log/audit/audit.log  
+
+```console
+ausearch -m avc -ts recent | audit2why
+journalctl -t setroubleshoot
+sealert -a /var/log/audit/audit.log
 ```
 
 * [BlueChi](https://github.com/containers/qm/pull/57)
@@ -68,40 +69,46 @@ repository enabled in order to provide `golang-github-cpuguy83-md2man` package.
 ```
 
 ## How the OOM Score Adjustment (`om_score_adj`) is used in QM
+
 The om_score_adj refers to the "Out of Memory score adjustment" in Linux operating systems. This parameter is used by the Out of Memory (OOM) killer to decide which processes to terminate when the system is critically low on memory.
 
 ### Why use `om_score_adj` in QM?
+
 By fine-tuning which processes are more likely to be terminated during low memory situations, critical processes can be protected, thereby enhancing the overall stability of the system. For instance only, ASIL (Automotive Safety Integrity Level) applications, which are critical for ensuring functional safety in automotive systems, will be preserved in case of low resources.
 
 ### OOM Score Adjustment in QM
 
-**Nested Containers**
-- All nested containers created inside QM will have their OOM score adjustment set to *750*.
+#### Nested Containers
+
+* All nested containers created inside QM will have their OOM score adjustment set to *750*.
 
-```bash
+```console
 $ cat /usr/share/qm/containers.conf | grep oom_score_adj
 oom_score_adj = 750
 ```
 
-**QM Process**
-- The QM process has a default OOM score adjustment value set to *500*, configured via the *qm.container* file.
+#### QM Process
+
+* The QM process has a default OOM score adjustment value set to *500*, configured via the *qm.container* file.
 
-```bash
+```console
 cat /usr/share/containers/systemd/qm.container | grep OOMScoreAdjust
 # OOMScoreAdjust=500
 ```
 
 ### ASIL Applications
+
 If we consider the example of ASIL (Automotive Safety Integrity Level) applications, which are essential for maintaining functional safety in automotive systems, their OOM score adjustment values can range from -1 to -1000. Setting the value to -1000 makes the process immune to the OOM killer. This ensures that ASIL applications are the last to be terminated by the OOM killer, thus prioritizing their operation even in low memory situations.
 
-**Highlights**
+#### Highlights
 
-- Nested Containers inside QM: OOM score adjustment set to 750. (/usr/share/qm/containers.conf)
-- QM Process: OOM score adjustment value set to 500, configured via the qm.container file.
-- ASIL Applications: Can explore a range from -1 to -1000, with -1000 making the process immune to the OOM killer.
+* Nested Containers inside QM: OOM score adjustment set to 750. (/usr/share/qm/containers.conf)
+* QM Process: OOM score adjustment value set to 500, configured via the qm.container file.
+* ASIL Applications: Can explore a range from -1 to -1000, with -1000 making the process immune to the OOM killer.
 
-**ASCII Diagram**
-```
+#### ASCII Diagram
+
+```txt
 +-------------------------------------------------------------+
 | The Priority Process of OOM Killer in the QM Context        |
 +-------------------------------------------------------------+
@@ -178,7 +185,7 @@ If your looking for contribute to the project use our [development README guide]
 
 ## RPM Mirrors
 
-Looking for a specific version of QM? 
-Search in the mirrors list below.  
+Looking for a specific version of QM?
+Search in the mirrors list below.
 
 [CentOS Automotive SIG - qm package - noarch](https://mirror.stream.centos.org/SIGs/9-stream/automotive/aarch64/packages-main/Packages/q/)
@@ -1,19 +1,23 @@
+# Developers documentation
+
+## Table of contents
+
 - [Building QM rpm manually with changes](#building-qm-rpm-manually-with-changes)
 - [Building CentOS AutoSD and QM manually](#building-centos-autosd-and-qm-manually)
 - [Useful Commands](#useful-commands)
   - [Installing software inside QM partition](#installing-software-inside-qm-partition)
   - [Removing software inside QM partition](#removing-software-inside-qm-partition)
   - [Copying files to QM partition](#copying-files-to-qm-partition)
-  - [Listing QM service](#Listing-QM-service)
-  - [List QM container via podman](#List-QM-container-via-podman)
-  - [Connecting to QM container via podman](#Connecting-to-QM-container-via-podman)
-  - [SSH guest CentOS Automotive Stream Distro](#SSH-guest-CentOS-Automotive-Stream-Distro)
-  - [Check if HOST and Container are using different network namespace](#Check-if-HOST-and-Container-are-using-different-network-namespace)
-  - [Debugging with podman in QM using --root](#Debugging-with-podman-in-QM)
+  - [Listing QM service](#listing-qm-service)
+  - [List QM container via podman](#list-qm-container-via-podman)
+  - [Connecting to QM container via podman](#connecting-to-qm-container-via-podman)
+  - [SSH guest CentOS Automotive Stream Distro](#ssh-guest-centos-automotive-stream-distro)
+  - [Check if HOST and Container are using different network namespace](#check-if-host-and-container-are-using-different-network-namespace)
+  - [Debugging with podman in QM using --root](#debugging-with-podman-in-qm)
 
-# Building QM rpm manually with changes
+## Building QM rpm manually with changes
 
-Building QM locally with changes for tests is a recommended practice,  
+Building QM locally with changes for tests is a recommended practice,
 especially for testing new features before submitting a pull request.
 
 **1.** Clone the repo
@@ -24,11 +28,11 @@ git clone https://github.com/containers/qm.git
 
 **2.** Change the Version macro in the spec file
 
-Set the QM version to a higher number from the current one.  
-For example, if today's QM version is 0.6.2, set it to 1.0 so that  
+Set the QM version to a higher number from the current one.
+For example, if today's QM version is 0.6.2, set it to 1.0 so that
 the RPM created is identifiable as yours.
 
-Use the following sed command to change the default Version (0, used for  
+Use the following sed command to change the default Version (0, used for
 automation) to the new version (1.0):
 
 ```bash
@@ -54,14 +58,14 @@ For example:
 VERSION=1.0 RPM_TOPDIR=/USER/rpmbuild make rpm
 ```
 
-# Building CentOS AutoSD and QM manually
+## Building CentOS AutoSD and QM manually
 
 During development, it is common to conduct integration tests to ensure your
-changes work well with other components within the overall solution.  
-In our case, it's best to test against the CentOS Automotive Stream  
+changes work well with other components within the overall solution.
+In our case, it's best to test against the CentOS Automotive Stream
 Distribution (AutoSD) image.
 
-Once you have the new [RPM](#building-qm-rpm), follow these steps:
+Once you have the new [RPM](#building-qm-rpm-manually-with-changes), follow these steps:
 
 **1.** Make sure the new rpm is located in **/USER/rpmbuild/RPMS/**
 
@@ -82,18 +86,18 @@ createrepo .
 
 **3.** Clone the CentOS Automotive distro for the build
 
-Ensure you meet the requirements for the CentOS Automotive Stream by  
+Ensure you meet the requirements for the CentOS Automotive Stream by
 referring to [this link](https://sigs.centos.org/automotive/building/).
 
 The following commands will execute:
 
-  - Cleanups before a fresh build
-  - Remove old qcow2 images used (regular and ostree)
-  - Finally creates a new image (BASED ON target name, ostree or regular)
-    NOTE:
-     - The path for the new QM rpm file (/root/rpmbuild/RPMS/noarch) 
-     - extra_rpms - useful for debug (do not use spaces between packages or will break)
-     - ssh enabled
+- Cleanups before a fresh build
+- Remove old qcow2 images used (regular and ostree)
+- Finally creates a new image (BASED ON target name, ostree or regular)
+  NOTE:
+  - The path for the new QM rpm file (/root/rpmbuild/RPMS/noarch)
+  - extra_rpms - useful for debug (do not use spaces between packages or will break)
+  - ssh enabled
 
 ```bash
 dnf install podman -y
@@ -106,28 +110,29 @@ rm -f cs9-qemu-qmcontainer-ostree.x86_64.qcow2
 make cs9-qemu-qmcontainer-regular.x86_64.qcow2 'DEFINES=extra_repos=[{"id":"local","baseurl":"file:///root/rpmbuild/RPMS/noarch"}] extra_rpms=["qm-1.0","vim-enhanced","strace","dnf","gdb","polkit","rsync","python3","openssh-server","openssh-clients"] ssh_permit_root_login=true osname="autosd" ssh_permit_password_auth=true'
 ```
 
-Run the virtual machine, default user: root, pass: password.  
+Run the virtual machine, default user: root, pass: password.
 To change default values, use the [defaults.ipp.yml](https://gitlab.com/CentOS/automotive/sample-images/-/blob/main/osbuild-manifests/include/defaults.ipp.ym) file.
 
 ```bash
 ./runvm --nographics ./cs9-qemu-qm-minimal-regular.x86_64.qcow2
 ```
 
-# Useful Commands
+## Useful Commands
 
-## Installing software inside QM partition
+### Installing software inside QM partition
 
 ```bash
 dnf --installroot /usr/lib/qm/rootfs/ install vim -y
 ```
 
-## Removing software inside QM partition
+### Removing software inside QM partition
 
 ```bash
 dnf --installroot /usr/lib/qm/rootfs/ remove vim -y
 ```
 
-## Copying files to QM partition
+### Copying files to QM partition
+
 Please note: This process is only applicable for regular images.
 OSTree images are read-only, and any files must be included during the build process.
 
@@ -141,7 +146,7 @@ bash-5.1> ls /root
 file_to_be_copied
 ```
 
-## Listing QM service
+### Listing QM service
 
 ```bash
 [root@localhost ~]# systemctl status qm -l
@@ -169,22 +174,23 @@ e-id a83253ae278d7394cb38e975535590d7 --max-bytes 536870912 --max-fds 4096 --max
 -matches 16384 --audit
 ```
 
-## List QM container via podman
+### List QM container via podman
 
-```bash
+```console
 # podman ps
 CONTAINER ID  IMAGE       COMMAND     CREATED         STATUS         PORTS       NAMES
 a83253ae278d              /sbin/init  38 seconds ago  Up 38 seconds              qm
 ```
 
-## Connecting to QM container via podman
+### Connecting to QM container via podman
 
-```bash
+```console
 # podman exec -it qm bash
 bash-5.1#
 ```
 
-## SSH guest CentOS Automotive Stream Distro
+### SSH guest CentOS Automotive Stream Distro
+
 Make sure the CentOS Automotive Stream Distro Virtual Machine/Container is running with SSHD enabled
 and permits ssh connection from root user.
 
@@ -210,31 +216,31 @@ Example connecting from the terminal to the Virtual Machine:
 
 ```bash
 connect-to-VM-via-SSH> ssh root@127.0.0.1 \
-			-p 2222 \
-			-oStrictHostKeyChecking=no \
-			-oUserKnownHostsFile=/dev/null
+    -p 2222 \
+    -oStrictHostKeyChecking=no \
+    -oUserKnownHostsFile=/dev/null
 ```
 
-## Check if HOST and Container are using different network namespace
+### Check if HOST and Container are using different network namespace
 
-**HOST**
+#### HOST
 
-```bash
+```console
 [root@localhost ~]# ls -l /proc/self/ns/net
 lrwxrwxrwx. 1 root root 0 May  1 04:33 /proc/self/ns/net -> 'net:[4026531840]'
 ```
 
-**QM**
+#### QM
 
-```bash
+```console
 bash-5.1# ls -l /proc/self/ns/net
 lrwxrwxrwx. 1 root root 0 May  1 04:33 /proc/self/ns/net -> 'net:[4026532287]'
 ```
 
-## Debugging with podman in QM
+### Debugging with podman in QM
 
-```bash
+```console
 bash-5.1# podman --root /usr/share/containers/storage pull alpine
-Error: creating runtime static files directory "/usr/share/containers/storage/libpod":  
+Error: creating runtime static files directory "/usr/share/containers/storage/libpod":
 mkdir /usr/share/containers/storage: read-only file system
 ```
@@ -1,6 +1,6 @@
-An example of Android container running on top of kvm using quadlet and Wayland.
+# An example of Android container running on top of kvm using quadlet and Wayland
 
-```
+```console
 $ cat ~/.config/containers/systemd/android.container
 
 [Service]

@@ -135,7 +135,7 @@ The systemd drop-in feature allows users to extend or modify the configuration o
 
 In this example, we will add the AllowedCPU to set as 1 to qm.service that quadlet generated:
 
-**Create allowedcpus.conf**
+### Create allowedcpus.conf
 
 ```console
 cat  /etc/systemd/system/qm.service.d/allowedcpus.conf
@@ -147,19 +147,20 @@ AllowedCPUs=1
 
 ## Tools
 
-**qm-rootfs**
+### qm-rootfs
 
 Prints the qm rootfs location previously configured during setup.
 
-**qm-storage-settings**
+### qm-storage-settings
 
 Setup the initial QM configuration for storage using the follow config files and changes:
-  - ${ROOTFS}/etc/containers/storage.conf
-    - uncomment additionalimagestores
-    - add /var/lib/shared into additionalimagestores
-    - uncomment and set to true the option transient_store
-  - ${ROOTFS}/etc/containers/container.conf
-    - add [engine] and TMPDIR
+
+- `${ROOTFS}/etc/containers/storage.conf`
+  - uncomment `additionalimagestores`
+  - add `/var/lib/shared` into `additionalimagestores`
+  - uncomment and set to `true` the option `transient_store`
+- `${ROOTFS}/etc/containers/container.conf`
+  - add `[engine]` and `TMPDIR`
 
 ## SEE ALSO
 

@@ -3,20 +3,25 @@
 Free-From-Interference (FFI) means applications within the QM environment do not interfere with applications in the ASIL environment.
 
 ## Memory
-QM environment will allocates 90% or greater of memory of the system. 
+
+QM environment will allocates 90% or greater of memory of the system.
 Launch application in the ASIL environment that requires > 10% of memory.
 Make sure swap is turned off.
 Make sure OOM Killer kills QM applications.
 
 Example:
-- memory/ASIL/20_percent_cpu_eat.c 
-- memory/QM/90_percent_cpu_eat.c 
+
+- `memory/ASIL/20_percent_cpu_eat.c`
+- `memory/QM/90_percent_cpu_eat.c`
 
 ## Disk
+
 Try to allocate as maximum possible disk space.
 
 ## Sysctl
+
 Running as nested container inside QM attempt to change settings in the host level (ASIL).
 
 ## deny_set_scheduler
-A tool to test if set_scheduler() is denied inside QM partition
+
+A tool to test if `set_scheduler()` is denied inside QM partition