Alter cpusets mems/cpus to allow restoring on nodes with different amt of resources

[eliams]

Hi,

So I want to checkpoint a container on one host and restore it in another one by moving the image directory and using --force option to restore in another container.
So I created a container on host B but restoring failed on host A (checkpoint restore on same host work on both of them).
What I did:
Host B:
docker run -d busybox /bin/sh -c 'i=0; while true; do echo $i; i=$(expr $i + 1); sleep 1; done'
docker checkpoint --work-dir=< work dir > --image-dir=< image dir >

Host A:
docker create busybox
docker restore --work-dir=< work dir > --image-dir=< image dir copied from host b > --force=true < created container id >

restore logs: http://pastebin.com/jpnhuePH
I don't understand if the /bin/sh mentioned in the error is containers or hosts files.

My environment on both hosts:
Debian based with custom 4.0.0 kernel.
Docker version 1.9.0-dev, build 2919249, experimental
(I am using aufs as storage driver)
CRIU Version: 1.7 GitID: v1.7-45-g7ae72c6

Thanks for the help.

[xemul]

The /bin/sh mentioned there is container's file.

[eliams]

So if I get it right the file /bin/sh does not have the same size in containers created from host A and B (The size was saved during checkpoint on host B and then when trying to restore to a new created container on host A it find a different size for that file) ? If this is true how it can be possible since both containers are from the same busybox image ?

I tried to run and checkpoint the container on host A and restore on host B but I got a different error:
http://pastebin.com/6ELkxnX3

[xemul]

As far as the /bin/sh is concerned -- can you check manually the sizes of it in A and B?

As far as the 2nd error is concerned -- it's ERANGE, smth is written into cpuset.mems that is not a valid mask. What's in you cgroup.img file?

[eliams]

I was using a different version of the busybox image on both hosts. Using the same version fixed the first error, sorry for missing it.

Here's the cgroup.img file: https://github.com/eliams/tmp/raw/master/criu_image_01/cgroup.img
The all criu image directory is here: https://github.com/eliams/tmp/tree/master/criu_image_01

[xemul]

                            {
                                "name": "cpuset.cpus", 
                                "value": "0-7"
                            }, 

I guess your other box doesn't accept THAT many CPUs, right?

Well, yes. This is a live migration issue, I would say -- restoring an image on a host with different amount of resources would require some images modification.

You can try to workaround this issue by

1. decoding cgroup.img file into json format (using crit)
2. manually fixing the cpuset.cpus value to be OK for your target machine
3. encoding the json back into cgroup.img (using crit)

After this the restore should work OK.

[eliams]

You are right, I was doing the checkpoint on a machine with 8 cpu and restoring on a machine with 4.
Manually fixing the cpuset.cpus value worked.

Thanks !

[xemul]

You're welcome!

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.