Doc: Containers vs VMs

[ShalokShalom]

I am currently improving this page of documentation, and like to understand some aspects of it first myself, so I can do that.

My assumption is, that most people come from a world, that consists mainly of containers and VMs.
I did not read before, that there may be a difference between something like an application container, and a system container.

I struggle to understand, what a system container contains, that an application container doesn't.

As far as I am aware, a Docker container always contains an operating system - including a file system, a certain userland, and the application itself.

Only the kernel is left to the host. So now I am reading, that there is a distinction between two types of container, and the provided documentation is not really going into the details, what parts are shared and what not.

From the limited information provided, do I assume that the difference is simply, that system containers allow to run multiple applications side by side and within the same container.

So it basically groups applications, and is besides this, still a container?
A multi-app container?

In order to improve the understanding of this page, do I consider making a table in Markdown.
I also seek to retain the black text of the provided images, and that one disappears in dark mode.

The images in question are both svg with a transparent background.
We would need to rerender them with an opaque background, to work.

[simos]

A Linux system is made of the Linux kernel, an init process (mother of all processes) and the rootfs (root filesystem). The Linux kernel boots up, finishes the core kernel initializations and then runs the init process from the rootfs. From then on, several processes are started and you have a working Linux system. You can view all these processes as a process tree, processes started by other processes, all leading to the initial mother process. One example of init process is systemd, which makes sense for desktop Linux systems and servers.

Is it possible to have more than one Linux system on a Linux system? On a running Linux system you could launch a init-compatible process on a separate rootfs. Well, these are the containers. In practice you would need to separate one such running process tree from another so they do not interfere with each other. This is performed using a Linux kernel feature called namespaces.

One application of containers is the application containers (pun intended), like Docker. The init has been optimized so that it runs only the specific things that you have described in the Dockerfile. When it is done running, the application container stops.

Another application of containers is the system containers, like incus. You have many options for a rootfs that much most of the Linux distributions. They also run the stock init of these Linux distributions. You launch a system container and it stays running until you shut it down. Just like with a Linux system or a VM.

Can you mess up the distinction even further? Sure you can. You can take Docker, replace the init process with something else and you get system containers.

All in all, with containers you are reusing the running Linux kernel to launch new process trees of different rootfs. The Linux kernel has a very stable API so any Linux distribution should work. The namespaces are separating each process tree for messing with each other. A container never includes a Linux kernel. A container expects that a Linux kernel is already running.

Namespaces can also be nested. You can run a namespace in a namespace. This means that you can run a container within a container. An incus server inside an incus container. Docker inside an incus container.

[ShalokShalom]

So, the difference between a system container and an application container is, as assumed, that the system container runs the native init system, and the application container has a modified init system, that is limited to only one process.

That is unless we use namespaces, which allow us to run multiple processes in one Docker container.

Is this correct so far? 😃

So then:

What can I do with a system container from Incus, that a Docker container with multiple namespaces can not? Is there any difference?

Can you take a look at my proposal, and tell me, if you think it is accurate, please 👍🏻

[simos]

So, the difference between a system container and an application container is, as assumed, that the system container runs the native init system, and the application container has a modified init system, that is limited to only one process.

I would say that a system container is like a typical Linux system. You boot it up, it gets ready, and it keeps running. If you want it to stop, you would shut it down like you do with a Linux system. I would not consider the init to be a differentiating factor; with system containers it is easier to reuse anything that is provided by the Linux distro.

On the other hand, an application container has a sole purpose to run a set of instructions to either perform a task and stop running at the end of it. Or it could be a service like a Web server that keeps running until you send a command to stop the Web server. When the Web server stops, this stops the container. You could create a Dockerfile that runs several services (like Web server, FTP server). It is due to best practices that you would keep separate tasks between application containers.

That is, an application container is task-oriented. The task finishes and the application container stops. A system container is a Linux system. You can keep one running without doing any task, you can run a task and then another task. The system container stops when you shut it down.

That is unless we use namespaces, which allow us to run multiple processes in one Docker container.

About 15 years ago there was this need to add support for containers to the Linux kernel. There was some talk to add specific container support. There was however another view to add new generic Linux kernel features that could be used for containers but also for possible other needs that do not relate to containers. That is, try to break down the requirements into a set of primitives. One such primitive is the namespaces, which allows to isolate processes, devices, etc (this process does not know that this other process exists. Another is cgroups, which allows to put limits on processes (i.e. a process cannot use more that X memory).

Any Linux container implementation would make use of namespaces, cgroups. Apart from incus, Docker uses both of them as well.

Is this correct so far? 😃

So then:

What can I do with a system container from Incus, that a Docker container with multiple namespaces can not? Is there any difference?

Both Incus and Docker use namespaces. The high-level difference between the two is that Incus is a Linux system while Docker is task-oriented.

Can you take a look at my proposal, and tell me, if you think it is accurate, please 👍🏻

Have a look at my PR. Take them as a suggestion. I would expect that it might require further changes in the wording.