[WIP] Initial support for containers

[angelnu]

Description

Adds support to build Open Container Initiative ( as Docker) containers.

Related Issue

#786

Types of changes

• Docs change / refactoring / dependency upgrade
• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Alternate Designs

• building just the tarball and keeping the docker wrapper in current angelnu/ccu git repo.
• extract files from ova images using qemu tools

Both are circumvention since they would patch RaspberryMatic after it has been built which makes very difficult not running in broken builds.

Possible Drawbacks

Kernel headers must be kept as old as possible to support running on host with older kernels. For example the Sinology NAS is still using kernel 4.4 which is the oldest currently supported by buildroot.

The old headers are needed in the OCI images AND the multilib. Currently I did not create new definitions for multilib since I do not see what problem would be using the older headers there but it could be done if anyone sees an issue.

Verification Process

Started the container on my amd64 workstation which uses kernel 4.19. All init error problems are gone and the UI comes up sucesfully. I still need to test with actual RF devices and build&test on arm.

Beyond that a CI will need to be contributed that uploads to a container repository. I propose to use https://quay.io/repository/angelnu/raspberrymatic (since docker hub new policy is not great)

Release Notes

• Container images available for amd64, arm and arm64

Contributing checklist

• My code follows the code style of this project.
• I have read the CONTRIBUTING and LICENSE document.
• I fully agree to distribute my changes under Apache 2.0 license.

[angelnu]

Build command

make PRODUCT=raspmatic_oci_amd64 build -j 8

Run command

docker run -ti --rm --name test -v ccu_data:/usr/local -p 8080:80 raspberrymatic:latest <- This is without any RF device yet

Boot output

Identifying onboard hardware: oci, OK
Initializing ZRAM Swap: SKIP
Initializing RTC Clock: no hardware found
Initializing urandom: SKIP
Checking for Factory Reset: not required
Checking for Backup Restore: not required
Initializing System: OK
Starting logging: OK
Identifying Homematic RF-Hardware: HmRF: none, HmIP: none, OK
Updating Homematic RF-Hardware: no GPIO/USB connected RF-hardware found
Starting bluetooth: disabled
Starting network: SKIP
Starting chrony: SKIP
Preparing start of hs485d: no Hm-Wired hardware found
Starting xinetd: OK
Starting eq3configd: OK
Starting lighttpd: OK
Starting ser2net: no configuration file
Starting ssdpd: OK
Starting NUT services: disabled
Initializing Third-Party Addons: OK
Starting LGWFirmwareUpdate: ...OK
Setting LAN Gateway keys: OK
Starting hs485d: no Hm-Wired hardware found
Starting multimacd: not required
Starting rfd: no BidCos-RF hardware found
Starting HMIPServer: ...OK
Starting ReGaHss: .OK
Starting Third-Party Addons: OK
Starting crond: OK
Setup onboard LEDs: booted, OK

Known bug:

Rega starts but is not serving pages

/var/log/messages

Jan  3 04:40:34 0548f20a4f50 daemon.err xinetd[196]: Unable to read included directory: /etc/config/xinetd.d [file=/etc/xinetd.conf] [line=14]
Jan  3 04:40:34 0548f20a4f50 daemon.crit xinetd[196]: 196 {init_services} no services. Exiting...
Sep 24 00:57:04 0548f20a4f50 local0.err ReGaHss: ERROR: original file not exists, so try load new file. [LoadOM():iseDOM.cpp:2610]
Jan  1 01:00:04 0548f20a4f50 local0.err ReGaHss: ERROR: new file not exists, so try load bak file. [LoadOM():iseDOM.cpp:2622]
Jan  1 01:00:00 0548f20a4f50 local0.err ReGaHss: ERROR: failed open file= /etc/config/homematic.regadom.bak [LoadFromFile():iseDOMpersist.cpp:78]
Nov  1 04:17:00 0548f20a4f50 local0.err ReGaHss: ERROR: (202) failed! Already in map! [Insert():iseDOMobj.cpp:506]
Nov  1 04:24:44 0548f20a4f50 local0.err ReGaHss: ERROR: IseAddFavorite failed! ID=202 [IseAddFavorite():iseDOM.cpp:644]
Nov  1 04:17:00 0548f20a4f50 local0.err ReGaHss: ERROR: (203) failed! Already in map! [Insert():iseDOMobj.cpp:506]
Nov  1 04:24:44 0548f20a4f50 local0.err ReGaHss: ERROR: IseAddFavorite failed! ID=203 [IseAddFavorite():iseDOM.cpp:644]
Nov  1 04:17:00 0548f20a4f50 local0.err ReGaHss: ERROR: (204) failed! Already in map! [Insert():iseDOMobj.cpp:506]
Nov  1 04:24:44 0548f20a4f50 local0.err ReGaHss: ERROR: IseAddFavorite failed! ID=204 [IseAddFavorite():iseDOM.cpp:644]

[jens-maus]

For of all: Thanks for this great contribution! Only a first small comment without having looked deeply yet into your changes:

Please rebase your PR onto a dedicated branch in your fork and please provide permissions for me (maintainer permissions) so that I can directly send in changes to this PR, because my plan is to get things done here first by submitting own changes to your PR before I will accept this PR. This will make things way easier!

EDIT:
And regarding publishing docker images yourself under quay.io: Please refrain from doing so at the moment! I want to publish a raspberrymatic docker only when it is done and finished and not confuse users by wild Raspberrymatic images appearing somewhere. In addition, to not confuse users I want to publish the docker images under my user-name to give this a more official nature. I hope you agree. And regarding docker hub: what's wrong with it?

[angelnu]

@jens-maus - I sent you an invite to add you to my fork where I am preparing the OCI feature. Once you accept it I can make you admin there. An alternative is that you create the the branch in your repo and give me access to it. I have no preferences :-)

I have no issues with the docker images being in your account but I need one to test the CI steps and moving the image to other computers to test. Ideally you would create one repo and make me (at least temporally) push rights. As alternative I would rename it to a less obvious name.

The reason IMO against Docker Hub is - https://www.docker.com/blog/docker-hub-image-retention-policy-delayed-and-subscription-updates/ . This pull consumption tier has already hit me over the last months and force me to log in with my account which is not so easy/secure when I am deploying a new Kubernetes cluster or testing a container in a server I do want to trust. Also people in my house that did not have an account where forced to create one. They can be mitigated but they add friction. Also Docker require projects to be excluded from this to be not only Open Source but not commercial. Quay.io is one of the alternatives people are moving to. I personally like their security scanner and support to keep open source projects in the free tier without additional conditions/processes.

EDIT: I have created a branch oci in my fork and removed the commits from master. I am not aware of an option to update the PR from to use the new branch, only the to - https://docs.github.com/en/free-pro-team@latest/github/collaborating-with-issues-and-pull-requests/changing-the-base-branch-of-a-pull-request So I would need to re-create the PR.

[jens-maus]

Yes, please recreate the PR then. I will then close that one here afterwards

And I don't need general access to your fork. Just make sure that in the new PR you have enabled maintainer access to the branch you created the PR from. Then I can directly commit to your branch without having to have general access to your while fork.

And regarding a container registry. You are right dockerhuv does not seem to be the right Place anymore. Too bad they changed their policy that much. However I would prefer to use the new GitHub docker registry GitHub provides since a while because it seem to integrate nicely with GitHub actions and the rest of the GitHub infrastructure. So as long as you develop this feature feel free to use your private quay.io registry. But for the final RaspberryMatic docker implementation I would like to use the GitHub registry, if possible.