No output with crictl checkpoint option

[neskandani]

Hi,

I am trying to checkpoint a container using crictl "checkpoint" option (sudo crictl checkpoint containerID). I was expecting to get a zip file containing the images generated by CRIU. Also I was expecting the container to stop. Unfortunately, I get only the container ID as the output of the command with no other result. Also, the container is still running after running the checkpoint command.

I am running the experiment on a k8s cluster(v1.27.4) with two Ubuntu22.04.2 nodes (kernel version: 5.15.0-1042-azure). The container runtime of the nodes is cri-o://1.26.3. I have also enabled the CRIU support for the runtime (enable_criu_support = true) and enabled the container checkpoint feature gate for the kube-apiserver (--feature-gates=ContainerCheckpoint=true)

[SergeyKanzhelev]

cc: @adrianreber

[SergeyKanzhelev]

Would be great if you can attach what you see with --debug flags. Also CRI-O logs will help.

[neskandani]

Sure, here is the output with the debug flag:

naf@karajanWorkerVM:~$ sudo crictl --debug checkpoint c9623d9b088e9
DEBU[0000] get runtime connection
DEBU[0000] CheckpointContainerRequest: &CheckpointContainerRequest{ContainerId:c9623d9b088e9,Location:,Timeout:0,}
c9623d9b088e9

I can see the pod log in /var/log/pods/pod_ID, but /var/log/crio/pods is empty.

[SergeyKanzhelev]

I meant cri-o logs from the runtime itself. Maybe something like journalctl -u crio

Also it is strange that the export location is empty. Are you passing it as an argument?

[adrianreber]

Please take a look at how CRI-O uses the command in its tests

https://github.com/cri-o/cri-o/blob/main/test/checkpoint.bats

I think the previous comment is correct you need to specify the location. The --export parameter is required.

That the container is not stopped is as expected by the KEP describing forensic container checkpointing.

[neskandani]

I meant cri-o logs from the runtime itself. Maybe something like journalctl -u crio

Also it is strange that the export location is empty. Are you passing it as an argument?

Sure, here is the log from the crio service:
logs.txt

I thought "export" is only an option. Previously, I tried to pass a directory path as an argument of this option, but I saw no result. Now (thanks to @adrianreber's pointer) I tried with passing a zip file path (sudo crictl checkpoint --export ~/checkpoints/opcua_checkpoints.tar 562e0082e53bd) and it worked! Thank you both a lot for the quick responses :)

[SergeyKanzhelev]

Filed: #1235