Skip to content
This repository has been archived by the owner on Aug 14, 2020. It is now read-only.

spec: add ExitPolicy type in pod manifest. #500

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion examples/pod_runtime.json
Original file line number Diff line number Diff line change
Expand Up @@ -102,5 +102,6 @@
"name": "ip-address",
"value": "10.1.2.3"
}
]
],
"exitPolicy": "onAnyFailure"
}
1 change: 1 addition & 0 deletions schema/pod.go
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ type PodManifest struct {
Isolators []types.Isolator `json:"isolators"`
Annotations types.Annotations `json:"annotations"`
Ports []types.ExposedPort `json:"ports"`
ExitPolicy types.ExitPolicy `json:"exitPolicy"`
}

// podManifest is a model to facilitate extra validation during the
Expand Down
57 changes: 57 additions & 0 deletions schema/types/exitpolicy.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
// Copyright 2015 The appc Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package types

import (
"encoding/json"
"fmt"
)

type ExitPolicy string

var validPolicies = map[ExitPolicy]struct{}{
"untilAll": struct{}{},
"onAny": struct{}{},
"onAnyFailure": struct{}{},
}

type exitPolicy ExitPolicy

func (e *ExitPolicy) UnmarshalJSON(data []byte) error {
var ep exitPolicy
if err := json.Unmarshal(data, &ep); err != nil {
return err
}
ee := ExitPolicy(ep)
if err := ee.assertValid(); err != nil {
return err
}
*e = ee
return nil
}

func (e ExitPolicy) MarshalJSON() ([]byte, error) {
if err := e.assertValid(); err != nil {
return nil, err
}
return json.Marshal(exitPolicy(e))
}

func (e ExitPolicy) assertValid() error {
if _, ok := validPolicies[e]; !ok {
return fmt.Errorf("invalid exit policy %q", string(e))
}
return nil
}
34 changes: 34 additions & 0 deletions schema/types/exitpolicy_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
// Copyright 2015 The appc Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package types

import (
"testing"
)

func TestGoodExitPolicy(t *testing.T) {
for e := range validPolicies {
if err := e.assertValid(); err != nil {
t.Errorf("good exit policy failed: %v", err)
}
}
}

func TestBadExitPolicy(t *testing.T) {
e := ExitPolicy("bad")
if err := e.assertValid(); err == nil {
t.Errorf("bad exit policy valid: %v", err)
}
}
7 changes: 6 additions & 1 deletion spec/pods.md
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,8 @@ JSON Schema for the Pod Manifest, conforming to [RFC4627](https://tools.ietf.org
"name": "ftp",
"hostPort": 2121
}
]
],
"exitPolicy": "onAnyFailure"
}
```

Expand Down Expand Up @@ -179,3 +180,7 @@ JSON Schema for the Pod Manifest, conforming to [RFC4627](https://tools.ietf.org
* **ports** (list of objects, optional) list of ports that SHOULD be exposed on the host.
* **name** (string, required, restricted to the [AC Name](#ac-name-type) formatting) name of the port to be exposed on the host. This field is a key referencing by name ports specified in the Image Manifest(s) of the app(s) within this Pod Manifest; consequently, port names MUST be unique among apps within a pod.
* **hostPort** (integer, required) port number on the host that will be mapped to the application port.
* **exitPolicy** (string, optional) a string that specify the exit policy of the pod, if left empty, then it's up to ACE to choose the default behavior. Valid values are:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First: Kubernetes is assuming "untilAll", and we resisted adding this for lack of really concrete use-cases. My instinct is that it SOUNDS cool, but isn't that useful IRL. As far as I know we have no such equivalent internally. If any app container exits with failure, we know the pod is doomed to fail, but we let the other containers finish.

But I'm going to assume you have a concrete set of use-cases that justify this (you should write them down in this PR description) or else you would not be adding hypothetical complexity.

I just went to refresh on the state of the spec, and I realize there is no (what kubernetes calls) restartPolicy. Is this supposed to be an analog of that? I think it's interesting to contrast the approaches.

Kubernetes defines:

  • RestartAlways: Always restart app containers, regardless of exit code. The pod can only terminate in Failure if the runtime decides that it is not viable (hardware failure, machine drain, etc).
  • RestartOnFailure: Restart containers if and only if they exited with a non-zero code. The pod's terminal state is the worst-of any container's terminal state.
  • RestartNever: Never intentionally restart containers. The pod's terminal state is the worst-of any container's terminal state.

Superficially kube's RestartAlways feels the same as untilAll here. But here's the rub - the definition of untilAll doesn't actually say anything about restart. Is that part of the policy here or is that governed somewhere else that I am not seeing?

I'll not write much more now, because I have asked enough questions that I am probably attacking a straw man.

From a functional POV I think the concepts that matter to a user are "when does my container get restarted?" and "what does that mean for the fate of my pod?", but this only answers the latter, and only partially.

From an API usability POV I think it might be clearer to express these things "in the positive". E.g. I think a "RunPolicy" would be clearer (RunForever, RunToCompletion, RunOnce), and I sort of wish Kubernetes had done it that way.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@thockin Actually that's part of the plan to implement k8s' restart policy in rkt

Basically as we are using systemd to launch rkt pods, my original plan is to use systemd's restart policy, and combined with this pod exit policy.

But I see your point, and we actually shouldn't make something just to ease the implementation... I am thinking to change this to restart policy, and implement it in the runtime itself. Thanks!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Think a little bit more on this, and I found that a RestartPolicy would imply that the runtime is long running, otherwise if the runtime get's killed, nothing can enforce the restart policy (e.g. kill a pod after killing kubelet, pod is not restarted)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any policy around exit/restart needs something to babysit, right? There's not a way (AFAIK) to tell the OS to kill process B when process A dies (short of SIGCHLD which is a stretch)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can do that with systemd service's dependency though.

My point is if the thing(or runtime) that launches the pod is not PID1, then the restart policy will not be enforced in some cases.
But maybe that's ok for now as we can limiting the scope of the restart policy, e.g. we assume the runtime is always there, and we don't consider what if the runtime gets killed.

People can just let PID1 to monitor the runtime, when the runtime fails, we treat it in a like a machine crash, and restart the runtime anyway(which will consequently restarts the pod).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've made the argument that "userspace is unreliable" in many arguments with Kernel folks, but their pushback (and rightly so) is "make it more reliable". There will always be corner cases, but there has to be a turtle at the bottom, and that turtle can't always be the kernel. In this case, I think kernel includes systemd - it really does fancy itself as important as the kernel.

So define the behavior you think is correct, and engineer towards a good enough answer.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I am happy with changing this to restart policy. Waiting for other maintainers' feedback.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please note that I am not saying you should change it to be like
kubernetes. Consider it in fresh light. I thing RestartPolicy is clearer
than ExitPolicy, but I think RunPolicy might be even better.

On Fri, Sep 25, 2015 at 4:40 PM, Yifan Gu notifications@github.com wrote:

In spec/pods.md
#500 (comment):

@@ -179,3 +180,7 @@ JSON Schema for the Pod Manifest, conforming to [RFC4627](https://tools.ietf.org

  • ports (list of objects, optional) list of ports that SHOULD be exposed on the host.
    • name (string, required, restricted to the AC Name formatting) name of the port to be exposed on the host. This field is a key referencing by name ports specified in the Image Manifest(s) of the app(s) within this Pod Manifest; consequently, port names MUST be unique among apps within a pod.
    • hostPort (integer, required) port number on the host that will be mapped to the application port.
      +* exitPolicy (string, optional) a string that specify the exit policy of the pod, if left empty, then it's up to ACE to choose the default behavior. Valid values are:

Sure, I am happy with changing this to restart policy. Waiting for other
maintainers' feedback.


Reply to this email directly or view it on GitHub
https://github.com/appc/spec/pull/500/files#r40485441.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to have something than empty here :) Any thoughts/votes on
ExitPolicy vs RestartPolicy vs RunPolicy?
@jonboulle @vbatts @philips ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ExecPolicy ?
On Oct 15, 2015 5:59 PM, "Yifan Gu" notifications@github.com wrote:

In spec/pods.md
#500 (comment):

@@ -179,3 +180,7 @@ JSON Schema for the Pod Manifest, conforming to [RFC4627](https://tools.ietf.org

  • ports (list of objects, optional) list of ports that SHOULD be exposed on the host.
    • name (string, required, restricted to the AC Name formatting) name of the port to be exposed on the host. This field is a key referencing by name ports specified in the Image Manifest(s) of the app(s) within this Pod Manifest; consequently, port names MUST be unique among apps within a pod.
    • hostPort (integer, required) port number on the host that will be mapped to the application port.
      +* exitPolicy (string, optional) a string that specify the exit policy of the pod, if left empty, then it's up to ACE to choose the default behavior. Valid values are:

I'd like to have something than empty here :) Any thoughts/votes on
ExitPolicy vs RestartPolicy vs RunPolicy?
@jonboulle https://github.com/jonboulle @vbatts
https://github.com/vbatts @philips https://github.com/philips ?


Reply to this email directly or view it on GitHub
https://github.com/appc/spec/pull/500/files#r42186397.

* **untilAll** - the pod exits only when all the apps exit, no matter they are successful or not.
* **onAny** - the pod exits when any of the apps exits either successfully or unsuccessfully.
* **onAnyFailure** -the pod exits when any of the pod exits unsuccessfully, also the pod exits when there is no app running.