add hooks stdin test #589

liangchenye · 2018-03-05T13:35:32Z

Signed-off-by: Liang Chenye liangchenye@huawei.com

liangchenye · 2018-03-05T13:41:23Z

It is a WIP PR to test the container state that passed to hooks over 'stdin':

The state of the container MUST be passed to hooks over stdin so that they may do work appropriate to the current state of the container.

I try to use timeout 1 cat > output to collect stdin content and compare the output file to do the test.
One problem is, it is not possible for the 'poststop' hook to get the container state since the container would be deleted before 'poststop hook' been called.

Poststop
The post-stop hooks MUST be called after the container is deleted but before the delete operation returns. Cleanup or debugging functions are examples of such a hook.

alban · 2018-03-05T14:35:08Z

One problem is, it is not possible for the 'poststop' hook to get the container state since the container would be deleted before 'poststop hook' been called.

It's not possible to call funC state container-id and compare with the output in the hook but we could still check that the state has the correct id, bundle and annotations.

I had a look at runc but it always gives an empty "status" (opencontainers/runc#1057 & opencontainers/runc#1741) and as you pointed out, it does not call the hooks at the correct time (opencontainers/runc#1710).

Since the spec does not have a "status=deleted", I wonder what's the correct status to send on stdin in the poststop hook. Does the spec explain that?

liangchenye · 2018-03-05T14:39:58Z

No, if it is deleted, there won't be a 'status=delete' since:

Once a container is deleted its ID MAY be used by a subsequent container

I think we can not send a status on stdin in the poststop hook.

liangchenye · 2018-03-15T08:11:59Z

In the updated version, I check the 'ID', 'Bundle' and 'Annotations' of the container state in 'prestart', 'poststart' and 'poststop' hooks. The 'PID'，'status' and 'ociVersion' is unpredictable.

The 'poststop' hook is also checked here according to current spec requirement.

liangchenye · 2018-03-15T08:12:55Z

PTAL @q384566678 @wking @alban

wking · 2018-03-15T11:57:16Z

I think we can not send a status on stdin in the poststop hook.

This is opencontainers/runtime-spec#958. We may end up with a deleted state, but as it stands I think runtimes have to use stopped there.

The 'PID'，'status' and 'ociVersion' is unpredictable.

Depending on how you read the spec, ociVersion needs to match either the config version or the state version. I have opencontainers/runtime-spec#903 in flight to settle on the latter. But until runtime-spec cuts a release with a different/expanded state schema, I think we can safely use:

if state.Version != "1.0.0" && state.Version != "1.0.1" {
    // complain
}

The container process can check its PID, and then this test can compare with the states it recieved.

liangchenye · 2018-03-15T11:59:12Z

I'm working on the PID now.
About the spec version, my new update is just comparing the Version: rspecs.Version,

wking · 2018-03-15T12:13:43Z

validation/hooks_stdin.go

+	g.AddPreStartHook(rspecs.Hook{
+		Path: filepath.Join(bundleDir, g.Spec().Root.Path, "/bin/sh"),
+		Args: []string{
+			"sh", "-c", fmt.Sprintf("timeout -t 1 cat > %s", filepath.Join(outputDir, "prestart")),


Where is your timeout from? It's not in POSIX, and coreutils uses -k / --kill-after, not -t.

It is in the busybox 1.28, in the new rootfs-amd64.tar.gz.

It is in the busybox 1.28, in the new rootfs-amd64.tar.gz.

The lack of standardization is unfortunate, because hooks are called from the runtime namespace, so the version in the container doesn't matter. We could work around that with {bundle-path}/rootfs/bin/timeout here, but it's probably better to set Timeout in the hook structure and skip the timeout command.

thanks, the timeout in hooks is much more rational.

wking · 2018-03-15T12:15:46Z

validation/hooks_stdin.go

+			if err != nil {
+				return err
+			}
+			containerPid = state.Pid


I think this would be more reliable if we dropped the state call and collected this from the container process itself (where you currently just use true).

echo $$ will output the id inside the container. It is different with the state one - It is 'seen' by host.

echo $$ will output the id inside the container. It is different with the state one - It is 'seen' by host.

Ah right, you'd need the PID namespace too to translate, and that would break down for VM-based runtimes anyway. So I'm on board with your current state call.

liangchenye · 2018-03-15T12:18:20Z

I'll revoke my version test. It is wrong.

liangchenye · 2018-03-15T13:04:34Z

Updated.
The 'version' test is dropped. I don't find a good definition of an expected version in the runtime spec.
We can get a 'Version' from r.State just like what we do to PID. But I don't see the meaning of doing this.

PTAL @q384566678

wking · 2018-03-15T14:19:20Z

validation/hooks_stdin.go

+	}
+	for _, file := range []string{"prestart", "poststart", "poststop"} {
+		err := stdinStateCheck(outputDir, file, expectedState)
+		util.SpecErrorOK(t, err == nil, specerror.NewError(specerror.PosixHooksStateToStdin, fmt.Errorf("the state of the container MUST be passed to %q hook over stdin so that they may do work appropriate to the current state of the container", file), rspecs.Version), err)


You can probably drop "so that they...of the container" from the error message. That motivation is in the spec for folks who care, but you don't need it for the check.

zhouhao3 · 2018-03-16T08:25:39Z

validation/hooks_stdin.go

+	}
+
+	if !reflect.DeepEqual(state.Annotations, expectedState.Annotations) {
+		return fmt.Errorf("wrong annotations \"%v\" in the stdin of %s hook, expected \"%v\"", state.Annotations, hookName, state.Annotations)


The last parameter should be expectedState.Annotations.

Signed-off-by: Liang Chenye <liangchenye@huawei.com>

zhouhao3 · 2018-03-16T08:29:37Z

The following error occurred while I was testing:

TAP version 13
not ok 1 - the state of the container MUST be passed to "prestart" hook over stdin
  ---
  {
    "error": "wrong annotations \"map[]\" in the stdin of prestart hook, expected \"map[org.opencontainers.runtime-tools:hook stdin test]\"",
    "reference": "https://github.com/opencontainers/runtime-spec/blob/v1.0.0/config.md#posix-platform-hooks"
  }
  ...
not ok 2 - the state of the container MUST be passed to "poststart" hook over stdin
  ---
  {
    "error": "wrong annotations \"map[]\" in the stdin of poststart hook, expected \"map[org.opencontainers.runtime-tools:hook stdin test]\"",
    "reference": "https://github.com/opencontainers/runtime-spec/blob/v1.0.0/config.md#posix-platform-hooks"
  }
  ...
not ok 3 - the state of the container MUST be passed to "poststop" hook over stdin
  ---
  {
    "error": "wrong annotations \"map[]\" in the stdin of poststop hook, expected \"map[org.opencontainers.runtime-tools:hook stdin test]\"",
    "reference": "https://github.com/opencontainers/runtime-spec/blob/v1.0.0/config.md#posix-platform-hooks"
  }
  ...
1..3

liangchenye · 2018-03-16T08:34:29Z

@q384566678 which 'runc' version do you use? Annotations are supported in mine.

runc --version
...
commit: e9cdc3906f18...
...

zhouhao3 · 2018-03-16T08:36:19Z

runc --version
runc version 1.0.0-rc4+dev
commit: 91e979501348cb4cb13b5fb4437cc5d9ecd94b5d
spec: 1.0.0

zhouhao3 · 2018-03-16T08:43:08Z

It's ok when I update the runc.

zhouhao3 · 2018-03-16T08:43:26Z

LGTM

wking · 2018-03-16T13:28:22Z

It's ok when I update the runc.

We should really check status to notice runc failing becaue of opencontainers/runc#1710 and opencontainers/runc#1741.

liangchenye · 2018-03-19T02:14:21Z

My understanding of the related spec is:
prestart hook status SHOULD be 'created'.
poststart hook status SHOULD be 'running'.
poststop hook ~~status SHOULD not be 'creating/created/running/stopped',~~ a 'nil' status or a runtime defined status would be OK.

I'll add a new PR to check this.

##Prestart

The pre-start hooks MUST be called after the start operation is called but before the user-specified program command is executed. ...

##Poststart

The post-start hooks MUST be called after the user-specified process is executed but before the start operation returns....

##Poststop

The post-stop hooks MUST be called after the container is deleted but before the delete operation returns....

liangchenye force-pushed the stdin branch 4 times, most recently from 2b6d3bd to 56a596f Compare March 15, 2018 08:06

liangchenye changed the title ~~WIP: add hooks stdin test~~ add hooks stdin test Mar 15, 2018

liangchenye force-pushed the stdin branch from 56a596f to 8da9225 Compare March 15, 2018 11:54

liangchenye force-pushed the stdin branch from 8da9225 to 9e4685d Compare March 15, 2018 12:09

wking reviewed Mar 15, 2018

View reviewed changes

liangchenye force-pushed the stdin branch from 9e4685d to 20328c9 Compare March 15, 2018 12:58

wking reviewed Mar 15, 2018

View reviewed changes

liangchenye force-pushed the stdin branch from 20328c9 to a5bf578 Compare March 16, 2018 01:54

zhouhao3 reviewed Mar 16, 2018

View reviewed changes

add hooks stdin test

73964f2

Signed-off-by: Liang Chenye <liangchenye@huawei.com>

liangchenye force-pushed the stdin branch from a5bf578 to 73964f2 Compare March 16, 2018 08:31

zhouhao3 merged commit b3796e2 into opencontainers:master Mar 19, 2018

liangchenye mentioned this pull request Mar 19, 2018

check the status of the state passed to hooks over stdin #608

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add hooks stdin test #589

add hooks stdin test #589

liangchenye commented Mar 5, 2018

liangchenye commented Mar 5, 2018

alban commented Mar 5, 2018

liangchenye commented Mar 5, 2018

liangchenye commented Mar 15, 2018

liangchenye commented Mar 15, 2018

wking commented Mar 15, 2018

liangchenye commented Mar 15, 2018

wking Mar 15, 2018 •

edited

Loading

liangchenye Mar 15, 2018

wking Mar 15, 2018

liangchenye Mar 15, 2018

wking Mar 15, 2018

liangchenye Mar 15, 2018

wking Mar 15, 2018

liangchenye commented Mar 15, 2018

liangchenye commented Mar 15, 2018 •

edited

Loading

wking Mar 15, 2018

liangchenye Mar 16, 2018

zhouhao3 Mar 16, 2018

liangchenye Mar 16, 2018

zhouhao3 commented Mar 16, 2018 •

edited

Loading

liangchenye commented Mar 16, 2018

zhouhao3 commented Mar 16, 2018

zhouhao3 commented Mar 16, 2018

zhouhao3 commented Mar 16, 2018 •

edited by caniszczyk

Loading

wking commented Mar 16, 2018

liangchenye commented Mar 19, 2018 •

edited

Loading

add hooks stdin test #589

add hooks stdin test #589

Conversation

liangchenye commented Mar 5, 2018

liangchenye commented Mar 5, 2018

alban commented Mar 5, 2018

liangchenye commented Mar 5, 2018

liangchenye commented Mar 15, 2018

liangchenye commented Mar 15, 2018

wking commented Mar 15, 2018

liangchenye commented Mar 15, 2018

wking Mar 15, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

liangchenye commented Mar 15, 2018

liangchenye commented Mar 15, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zhouhao3 commented Mar 16, 2018 • edited Loading

liangchenye commented Mar 16, 2018

zhouhao3 commented Mar 16, 2018

zhouhao3 commented Mar 16, 2018

zhouhao3 commented Mar 16, 2018 • edited by caniszczyk Loading

wking commented Mar 16, 2018

liangchenye commented Mar 19, 2018 • edited Loading

wking Mar 15, 2018 •

edited

Loading

liangchenye commented Mar 15, 2018 •

edited

Loading

zhouhao3 commented Mar 16, 2018 •

edited

Loading

zhouhao3 commented Mar 16, 2018 •

edited by caniszczyk

Loading

liangchenye commented Mar 19, 2018 •

edited

Loading