Add --init option to `docker service create` #479

alexkreidler · 2017-08-28T20:46:17Z

Helps #51, but requires more work to add stack/compose v3 file support.
See moby/swarmkit#2350, and moby/moby#34529. We've got it all set in swarmkit, this should finish the CLI, and we're waiting on the final pieces in moby.

- What I did
Added a --init option to docker service create
- How I did it
Added an options string (flagInit), the swarm api field in opts.ToService(), and a boolean flag in newCreateCommand()

I also changed a line in the vendor code to add the Init option to the swarm container api. I understand I'm not supposed to do this, but I'll wait till they get a PR into moby/moby, and then do the vendoring properly.

- How to verify it
Run the tests for the service subcommand. Since the daemon doesn't support init for swarm yet, it won't actually do anything.
- Description for the changelog

Add --init option to docker service create

Waiting for the following PR to get merged

thaJeztah

I gave this a spin against a docker 17.09-rc2 daemon (which should have the SwarmKit changes), but it didn't seem to work;

Creating a service with --init looks to create the right request:

docker service create --init --name initservice nginx:alpine

Sep 20 10:29:30 moby root: time="2017-09-20T10:29:30.328450671Z" level=debug msg="Calling POST /v1.31/services/create"
Sep 20 10:29:30 moby root: time="2017-09-20T10:29:30.328573170Z" level=debug msg="form data: {\"EndpointSpec\":{\"Mode\":\"vip\"},\"Labels\":{},\"Mode\":{\"Replicated\":{}},\"Name\":\"initservice\",\"TaskTemplate\":{\"ContainerSpec\":{\"DNSConfig\":{},\"Image\":\"nginx:alpine@sha256:83f10f82722087e6944e0348b2e64a95baf247135de7c237f4dec7729a386d7f\",\"Init\":true},\"ForceUpdate\":0,\"Placement\":{\"Platforms\":[{\"Architecture\":\"amd64\",\"OS\":\"linux\"}]},\"Resources\":{\"Limits\":{},\"Reservations\":{}}}}"

(formatted for readability): I can see the Init to be passed to the daemon;

{
  "EndpointSpec": {
    "Mode": "vip"
  },
  "Labels": {
    
  },
  "Mode": {
    "Replicated": {
      
    }
  },
  "Name": "initservice",
  "TaskTemplate": {
    "ContainerSpec": {
      "DNSConfig": {
        
      },
      "Image": "nginx:alpine@sha256:83f10f82722087e6944e0348b2e64a95baf247135de7c237f4dec7729a386d7f",
      "Init": true
    },
    "ForceUpdate": 0,
    "Placement": {
      "Platforms": [
        {
          "Architecture": "amd64",
          "OS": "linux"
        }
      ]
    },
    "Resources": {
      "Limits": {
        
      },
      "Reservations": {
        
      }
    }
  }
}

However, I don't see the Init option set for the task backing the container:

docker container inspect  initservice.1.xsbsrewdxj0eq9sg2sa51p3im | grep Init
(no response)

Inspecting the service also does not show this information, which may be either because there's an issue there, or because docker service inspect does not show all information;

[
    {
        "ID": "3kju7dy0b7gz0hl8mf5yy93ub",
        "Version": {
            "Index": 90211
        },
        "CreatedAt": "2017-09-20T10:29:30.329124642Z",
        "UpdatedAt": "2017-09-20T10:29:30.329124642Z",
        "Spec": {
            "Name": "initservice",
            "Labels": {},
            "TaskTemplate": {
                "ContainerSpec": {
                    "Image": "nginx:alpine@sha256:83f10f82722087e6944e0348b2e64a95baf247135de7c237f4dec7729a386d7f",
                    "StopGracePeriod": 10000000000,
                    "DNSConfig": {}
                },
                "Resources": {
                    "Limits": {},
                    "Reservations": {}
                },
                "RestartPolicy": {
                    "Condition": "any",
                    "Delay": 5000000000,
                    "MaxAttempts": 0
                },
                "Placement": {
                    "Platforms": [
                        {
                            "Architecture": "amd64",
                            "OS": "linux"
                        }
                    ]
                },
                "ForceUpdate": 0,
                "Runtime": "container"
            },
            "Mode": {
                "Replicated": {
                    "Replicas": 1
                }
            },
            "UpdateConfig": {
                "Parallelism": 1,
                "FailureAction": "pause",
                "Monitor": 5000000000,
                "MaxFailureRatio": 0,
                "Order": "stop-first"
            },
            "RollbackConfig": {
                "Parallelism": 1,
                "FailureAction": "pause",
                "Monitor": 5000000000,
                "MaxFailureRatio": 0,
                "Order": "stop-first"
            },
            "EndpointSpec": {
                "Mode": "vip"
            }
        },
        "Endpoint": {
            "Spec": {}
        }
    }
]

thaJeztah · 2017-09-20T10:19:23Z

cli/command/service/create.go

@@ -57,6 +57,8 @@ func newCreateCommand(dockerCli command.Cli) *cobra.Command {
 	flags.SetAnnotation(flagDNSSearch, "version", []string{"1.25"})
 	flags.Var(&opts.hosts, flagHost, "Set one or more custom host-to-IP mappings (host:ip)")
 	flags.SetAnnotation(flagHost, "version", []string{"1.25"})
+	flags.BoolVar(&opts.init, flagInit, false, "Use an init inside each service container to forward signals and reap processes")
+	flags.SetAnnotation(flagInit, "version", []string{"1.30"})


This will probably need to be bumped to 1.33 (1.32 has shipped in a release)

thaJeztah · 2017-09-20T10:27:21Z

vendor/github.com/docker/docker/api/types/swarm/container.go

@@ -55,6 +55,7 @@ type ContainerSpec struct {
 	User            string                  `json:",omitempty"`
 	Groups          []string                `json:",omitempty"`
 	Privileges      *Privileges             `json:",omitempty"`
+	Init            bool                    `json:",omitempty"`


As commented in moby/moby#34529 (comment) - SwarmKit changes are merged in moby/moby, so can you open a pull request there to make the required changes in moby/moby?

(you, or @denverdino ) 😇

thaJeztah · 2017-09-20T10:40:38Z

Actually, nevermind, the daemon API of course doesn't yet put this property forward, so that's expected that it's non functional yet. 😅

thaJeztah · 2017-11-08T16:52:42Z

Bumping moby/moby in #679, which should bring in the required changes

leshik · 2018-03-23T16:14:32Z

Any chances we'll see this feature in near future?

codecov-io · 2018-05-31T13:48:50Z

Codecov Report

❗ No coverage uploaded for pull request base (master@722d08f). Click here to learn what that means.
The diff coverage is 0%.

@@            Coverage Diff            @@
##             master     #479   +/-   ##
=========================================
  Coverage          ?   48.52%           
=========================================
  Files             ?      199           
  Lines             ?    16409           
  Branches          ?        0           
=========================================
  Hits              ?     7962           
  Misses            ?     8033           
  Partials          ?      414

vdemeester · 2018-05-31T13:49:23Z

I am carrying this PR, opened upstream PR on moby and swarmkit

vdemeester · 2018-05-31T13:50:19Z

btw, design LGTM 👼

thaJeztah · 2018-05-31T14:11:22Z

Just so that I don't forget;

This option needs to be on docker service update as well
We need a test to verify that running docker service update without passing the --init flag preserves the existing value
We need a test (probably in moby/moby) to verify that the daemon option (which can set --init so that it's added by default) is used if the --init flag is not passed, on docker service create

vdemeester · 2018-05-31T14:39:14Z

service update is taken care of, with a unit test (to validate the spec update or not)
I'll add a test in moby for the daemon conf 😉

thaJeztah · 2018-06-07T23:30:00Z

Oh, last two bullets;

Add the new option to the --pretty output for docker service inspect;

cli/cli/command/formatter/service.go

Lines 19 to 142 in 7285241

    
           const serviceInspectPrettyTemplate Format = ` 
        
           ID:		{{.ID}} 
        
           Name:		{{.Name}} 
        
           {{- if .Labels }} 
        
           Labels: 
        
           {{- range $k, $v := .Labels }} 
        
            {{ $k }}{{if $v }}={{ $v }}{{ end }} 
        
           {{- end }}{{ end }} 
        
           Service Mode: 
        
           {{- if .IsModeGlobal }}	Global 
        
           {{- else if .IsModeReplicated }}	Replicated 
        
           {{- if .ModeReplicatedReplicas }} 
        
            Replicas:	{{ .ModeReplicatedReplicas }} 
        
           {{- end }}{{ end }} 
        
           {{- if .HasUpdateStatus }} 
        
           UpdateStatus: 
        
            State:		{{ .UpdateStatusState }} 
        
           {{- if .HasUpdateStatusStarted }} 
        
            Started:	{{ .UpdateStatusStarted }} 
        
           {{- end }} 
        
           {{- if .UpdateIsCompleted }} 
        
            Completed:	{{ .UpdateStatusCompleted }} 
        
           {{- end }} 
        
            Message:	{{ .UpdateStatusMessage }} 
        
           {{- end }} 
        
           Placement: 
        
           {{- if .TaskPlacementConstraints }} 
        
            Constraints:	{{ .TaskPlacementConstraints }} 
        
           {{- end }} 
        
           {{- if .TaskPlacementPreferences }} 
        
            Preferences:   {{ .TaskPlacementPreferences }} 
        
           {{- end }} 
        
           {{- if .HasUpdateConfig }} 
        
           UpdateConfig: 
        
            Parallelism:	{{ .UpdateParallelism }} 
        
           {{- if .HasUpdateDelay}} 
        
            Delay:		{{ .UpdateDelay }} 
        
           {{- end }} 
        
            On failure:	{{ .UpdateOnFailure }} 
        
           {{- if .HasUpdateMonitor}} 
        
            Monitoring Period: {{ .UpdateMonitor }} 
        
           {{- end }} 
        
            Max failure ratio: {{ .UpdateMaxFailureRatio }} 
        
            Update order:      {{ .UpdateOrder }} 
        
           {{- end }} 
        
           {{- if .HasRollbackConfig }} 
        
           RollbackConfig: 
        
            Parallelism:	{{ .RollbackParallelism }} 
        
           {{- if .HasRollbackDelay}} 
        
            Delay:		{{ .RollbackDelay }} 
        
           {{- end }} 
        
            On failure:	{{ .RollbackOnFailure }} 
        
           {{- if .HasRollbackMonitor}} 
        
            Monitoring Period: {{ .RollbackMonitor }} 
        
           {{- end }} 
        
            Max failure ratio: {{ .RollbackMaxFailureRatio }} 
        
            Rollback order:    {{ .RollbackOrder }} 
        
           {{- end }} 
        
           ContainerSpec: 
        
            Image:		{{ .ContainerImage }} 
        
           {{- if .ContainerArgs }} 
        
            Args:		{{ range $arg := .ContainerArgs }}{{ $arg }} {{ end }} 
        
           {{- end -}} 
        
           {{- if .ContainerEnv }} 
        
            Env:		{{ range $env := .ContainerEnv }}{{ $env }} {{ end }} 
        
           {{- end -}} 
        
           {{- if .ContainerWorkDir }} 
        
            Dir:		{{ .ContainerWorkDir }} 
        
           {{- end -}} 
        
           {{- if .ContainerUser }} 
        
            User: {{ .ContainerUser }} 
        
           {{- end }} 
        
           {{- if .ContainerMounts }} 
        
           Mounts: 
        
           {{- end }} 
        
           {{- range $mount := .ContainerMounts }} 
        
            Target:	{{ $mount.Target }} 
        
             Source:	{{ $mount.Source }} 
        
             ReadOnly:	{{ $mount.ReadOnly }} 
        
             Type:		{{ $mount.Type }} 
        
           {{- end -}} 
        
           {{- if .Configs}} 
        
           Configs: 
        
           {{- range $config := .Configs }} 
        
            Target:	{{$config.File.Name}} 
        
             Source:	{{$config.ConfigName}} 
        
           {{- end }}{{ end }} 
        
           {{- if .Secrets }} 
        
           Secrets: 
        
           {{- range $secret := .Secrets }} 
        
            Target:	{{$secret.File.Name}} 
        
             Source:	{{$secret.SecretName}} 
        
           {{- end }}{{ end }} 
        
           {{- if .HasResources }} 
        
           Resources: 
        
           {{- if .HasResourceReservations }} 
        
            Reservations: 
        
           {{- if gt .ResourceReservationNanoCPUs 0.0 }} 
        
             CPU:		{{ .ResourceReservationNanoCPUs }} 
        
           {{- end }} 
        
           {{- if .ResourceReservationMemory }} 
        
             Memory:	{{ .ResourceReservationMemory }} 
        
           {{- end }}{{ end }} 
        
           {{- if .HasResourceLimits }} 
        
            Limits: 
        
           {{- if gt .ResourceLimitsNanoCPUs 0.0 }} 
        
             CPU:		{{ .ResourceLimitsNanoCPUs }} 
        
           {{- end }} 
        
           {{- if .ResourceLimitMemory }} 
        
             Memory:	{{ .ResourceLimitMemory }} 
        
           {{- end }}{{ end }}{{ end }} 
        
           {{- if .Networks }} 
        
           Networks: 
        
           {{- range $network := .Networks }} {{ $network }}{{ end }} {{ end }} 
        
           Endpoint Mode:	{{ .EndpointMode }} 
        
           {{- if .Ports }} 
        
           Ports: 
        
           {{- range $port := .Ports }} 
        
            PublishedPort = {{ $port.PublishedPort }} 
        
             Protocol = {{ $port.Protocol }} 
        
             TargetPort = {{ $port.TargetPort }} 
        
             PublishMode = {{ $port.PublishMode }} 
        
           {{- end }} {{ end -}} 
        
           `

Updates to the docs 😅

thaJeztah · 2018-06-07T23:36:18Z

cli/command/service/update_test.go

+	}
+	cspec := spec.TaskTemplate.ContainerSpec
+
+	// Update with --stop-signal=SIGUSR1


s/--stop-signal=SIGUSR1/--init=true/

thaJeztah · 2018-06-07T23:36:41Z

cli/command/service/update_test.go

+	updateService(nil, nil, flags, spec)
+	assert.Check(t, is.Equal(true, cspec.Init))
+
+	// Update without --stop-signal, no change


s/--stop-signal/--init/

thaJeztah · 2018-06-07T23:37:03Z

cli/command/service/update_test.go

+	updateService(nil, nil, flags, spec)
+	assert.Check(t, is.Equal(true, cspec.Init))
+
+	// Update with --stop-signal=SIGWINCH


s/--stop-signal= SIGWINCH/--init=false/

silvin-lubecki · 2018-06-08T15:26:34Z

Validate is complaining:

These files were changed:
 M vendor/github.com/docker/docker/api/types/swarm/container.go

vdemeester · 2018-06-08T15:34:32Z

Add the new option to the --pretty output for docker service inspect;

@thaJeztah actually there is quite some field from the ContainerSpec that are not showed in the --pretty (like TTY, …). I wonder if we should add all of them too…

thaJeztah · 2018-06-09T04:39:20Z

We should probably add more of those yes; but it's a good start to make sure that new fields are not missed

Moby vendoring was updated in another PR, so looks like we can continue on this one 👍

vdemeester

LGTM 😝

silvin-lubecki · 2018-06-11T13:38:36Z

test is complaining 😂

--- FAIL: TestPrettyPrintWithNoUpdateConfig (0.00s)
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xbaceb4]

goroutine 8 [running]:
testing.tRunner.func1(0xc4203fc1e0)
	/usr/local/go/src/testing/testing.go:742 +0x29d
panic(0xd2d540, 0x13dd220)
	/usr/local/go/src/runtime/panic.go:502 +0x229
text/template.errRecover(0xc420175490)
	/usr/local/go/src/text/template/exec.go:137 +0x1d4
panic(0xd2d540, 0x13dd220)
	/usr/local/go/src/runtime/panic.go:502 +0x229
github.com/docker/cli/cli/command/formatter.(*serviceInspectContext).ContainerInit(0xc42008fa00, 0x0)
	/go/src/github.com/docker/cli/cli/command/formatter/service.go:381 +0x14
...

thaJeztah · 2018-06-11T18:26:25Z

docs/reference/commandline/service_update.md

-      --container-label-add list           Add or update a container label
-      --container-label-rm list            Remove a container label by its key
-      --credential-spec credential-spec    Credential spec for managed service account (Windows only)
+	  --args command                       Service command args


Looks like this whole file changed from using spaces to tabs; can you revert that change?

silvin-lubecki · 2018-06-13T08:57:01Z

A test is failing:

--- FAIL: TestUpdateInit (0.00s)
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
    panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xc36156]

thaJeztah

LGTM after the test is fixed

Signed-off-by: Timothy Higinbottom <timhigins@gmail.com> Signed-off-by: Vincent Demeester <vincent@sbr.pm>

thaJeztah · 2018-06-23T01:21:31Z

Follow up for the compose-file format; #1129

GordonTheTurtle added the status/0-triage label Aug 28, 2017

thaJeztah mentioned this pull request Sep 19, 2017

Cannot kill or detach a docker container using Ctrl-C or Ctrl-\ moby/moby#2838

Closed

vdemeester added status/1-design-review and removed status/0-triage labels Sep 20, 2017

vdemeester requested review from dnephin, tiborvass, vieux, thaJeztah and aaronlehmann September 20, 2017 09:53

thaJeztah added the status/needs-vendoring label Sep 20, 2017

thaJeztah requested changes Sep 20, 2017

View reviewed changes

thaJeztah mentioned this pull request Nov 6, 2017

Need documentation on new docker-compose feature "init" docker/docs#3149

Closed

thaJeztah mentioned this pull request May 30, 2018

Add 3.7 schema and add rollback_config to it docker/compose#5992

Merged

This was referenced May 31, 2018

Fix deepcopy.Copy panicing with BoolValue moby/swarmkit#2652

Merged

Add support for init on services moby/moby#37183

Merged

vdemeester force-pushed the init-option branch from a3f1d89 to b56eb76 Compare May 31, 2018 13:48

thaJeztah added status/2-code-review and removed status/1-design-review labels May 31, 2018

vdemeester force-pushed the init-option branch 2 times, most recently from b4d89ba to cbaf7d9 Compare May 31, 2018 14:38

thaJeztah requested changes Jun 7, 2018

View reviewed changes

vdemeester force-pushed the init-option branch from cbaf7d9 to a782c83 Compare June 8, 2018 08:54

vdemeester force-pushed the init-option branch from a782c83 to dece8a7 Compare June 8, 2018 15:39

alexkreidler requested a review from vdemeester as a code owner June 8, 2018 15:39

vdemeester changed the title ~~[WIP] Add --init option to docker service create~~ Add --init option to docker service create Jun 11, 2018

vdemeester force-pushed the init-option branch from dece8a7 to 6fe9552 Compare June 11, 2018 12:53

vdemeester approved these changes Jun 11, 2018

View reviewed changes

thaJeztah requested changes Jun 11, 2018

View reviewed changes

thaJeztah added impact/changelog and removed status/needs-vendoring labels Jun 11, 2018

vdemeester force-pushed the init-option branch from 6fe9552 to 24c7fb7 Compare June 12, 2018 07:13

thaJeztah approved these changes Jun 13, 2018

View reviewed changes

Add --init option to docker service create

ff13f03

Signed-off-by: Timothy Higinbottom <timhigins@gmail.com> Signed-off-by: Vincent Demeester <vincent@sbr.pm>

vdemeester force-pushed the init-option branch from 24c7fb7 to ff13f03 Compare June 14, 2018 11:50

vdemeester merged commit 70db8de into docker:master Jun 14, 2018

GordonTheTurtle added this to the 18.06.0 milestone Jun 14, 2018

techieshark mentioned this pull request Jun 23, 2018

compose file version 3.0 removed support for init docker/compose#5049

Closed

vdemeester mentioned this pull request Jun 25, 2018

Add init support in 3.7 schema #1129

Merged

This was referenced Jul 10, 2018

[epic] add more options to service create / service update moby/moby#25303

Open

Add --init options to service create moby/moby#34529

Closed

albers mentioned this pull request Jul 13, 2018

Add bash completion for service create|update --init #1210

Merged

thaJeztah mentioned this pull request Jul 13, 2018

[18.06] backport bash completion for service create|update --init #1211

Merged

jayjun mentioned this pull request Jan 27, 2019

Start long running containers with --init dokku/dokku#3422

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add --init option to `docker service create` #479

Add --init option to `docker service create` #479

alexkreidler commented Aug 28, 2017 •

edited

Loading

thaJeztah left a comment

thaJeztah Sep 20, 2017

thaJeztah Sep 20, 2017

thaJeztah Sep 20, 2017

thaJeztah commented Sep 20, 2017

thaJeztah commented Nov 8, 2017

leshik commented Mar 23, 2018

codecov-io commented May 31, 2018 •

edited

Loading

vdemeester commented May 31, 2018

vdemeester commented May 31, 2018

thaJeztah commented May 31, 2018

vdemeester commented May 31, 2018

thaJeztah commented Jun 7, 2018

thaJeztah Jun 7, 2018

thaJeztah Jun 7, 2018

thaJeztah Jun 7, 2018

silvin-lubecki commented Jun 8, 2018

vdemeester commented Jun 8, 2018

thaJeztah commented Jun 9, 2018

vdemeester left a comment

silvin-lubecki commented Jun 11, 2018

thaJeztah Jun 11, 2018

vdemeester Jun 11, 2018

silvin-lubecki commented Jun 13, 2018

thaJeztah left a comment

thaJeztah commented Jun 23, 2018

Add --init option to docker service create #479

Add --init option to docker service create #479

Conversation

alexkreidler commented Aug 28, 2017 • edited Loading

thaJeztah left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

thaJeztah commented Sep 20, 2017

thaJeztah commented Nov 8, 2017

leshik commented Mar 23, 2018

codecov-io commented May 31, 2018 • edited Loading

Codecov Report

vdemeester commented May 31, 2018

vdemeester commented May 31, 2018

thaJeztah commented May 31, 2018

vdemeester commented May 31, 2018

thaJeztah commented Jun 7, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

silvin-lubecki commented Jun 8, 2018

vdemeester commented Jun 8, 2018

thaJeztah commented Jun 9, 2018

vdemeester left a comment

Choose a reason for hiding this comment

silvin-lubecki commented Jun 11, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

silvin-lubecki commented Jun 13, 2018

thaJeztah left a comment

Choose a reason for hiding this comment

thaJeztah commented Jun 23, 2018

Add --init option to `docker service create` #479

Add --init option to `docker service create` #479

alexkreidler commented Aug 28, 2017 •

edited

Loading

codecov-io commented May 31, 2018 •

edited

Loading