sending a kill TERM signal does not result in containers being shut down

[jbulchergist]

I'm using fig up --no-recreate to run a container, which works as expected. Fig apparently gets two services in the process (weird):

# ps aux|grep fig                                                                                                                         
root      9341  0.5  0.0  12960  2016 pts/2    S+   13:19   0:00 fig up --no-recreate
root      9342  2.0  0.8 134992 16408 pts/2    Sl+  13:19   0:00 fig up --no-recreate

When I perfirm fig stop, the container goes down as expected, but if I perform kill 9341, the container does not go down even though fig does:

# kill 9341 && ps aux|grep fig
root      9353  0.0  0.0  11744   920 pts/3    S+   13:22   0:00 grep --color=auto fig
# fig ps
     Name        Command   State   Ports 
----------------------------------------
jbulcher_app_1   /run.sh   Up 

If the container does not shut down with a kill signal, it isn't possible to integrate it into upstart/systemd, is it?

[dnephin]

If you're trying to integrate with an init system, I would use fig up -d instead. To shut it down use fig stop.

Killing the fig process doesn't handle signals and forward them to all the running docker containers. I'm not sure if it should.

[jbulchergist]

If fig is to be used in init systems it seems very desirable - and it seems important regardless. Reading up on my signals, I figured out that Ctrl-C sends the SIGINT signal, whereas upstart just sends SIGTERM. Is there a good reason that fig should handle SIGINT gracefully, and not SIGTERM?

My argument that fig should handle SIGTERM graceful is that fig is used to control services. If someone sends a SIGTERM to fig, they would expect the service to shut down - not just fig. That breaks the service control, and is unexpected. Furthermore, that SIGINT is capable of properly terminating the service when SIGTERM fails seems to me completely... wonky.

[dnephin]

@jbulchergist see #741, we'd actually like to change the behaviour of SIGINT, and have it not shutdown containers either. Having it shutdown containers on SIGTERM does make more sense than SIGINT.

[jbulchergist]

What data containers are you using? fig up doesn't shut down on me when I'm using a busybox based data container, nor does the data container continue running as a separate process:

# fig ps
       Name                    Command              State          Ports        
-------------------------------------------------------------------------------
tikiwiki_apache_1    /usr/local/sbin/run.sh         Up       0.0.0.0:80->80/tcp 
tikiwiki_data_1      /bin/sh -c true                Exit 0                      
tikiwiki_mariadb_1   /docker-entrypoint.sh mysqld   Up       3306/tcp 

The data Dockerfile:

FROM busybox
VOLUME /var/lib/mysql
CMD true

[dnephin]

I believe there is actually a race condition as well. If the log tailing starts after the container is already shut down then it will miss it. #603 (comment)

[jbulchergist]

Is that due to a broken container? Rather than changing the behavior of fig up, it would be better to change the behavior of fig when it attempts to get logs. I've had trouble before with fig logs; I have not had trouble with fig up.

[dnephin]

Part of this was fixed by #1754, but there might still be an issue with SIGTERM.

[dnephin]

When this is implemented https://github.com/buildkite/docker-signal-test has some test cases that can be implemented to verify SIGTERM behaviour.