Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AWS Remote exec script exits prematurely in v0.6.13, not it 0.6.12 #5699

Closed
byrnedo opened this issue Mar 17, 2016 · 4 comments · Fixed by #5751
Closed

AWS Remote exec script exits prematurely in v0.6.13, not it 0.6.12 #5699

byrnedo opened this issue Mar 17, 2016 · 4 comments · Fixed by #5751

Comments

@byrnedo
Copy link

byrnedo commented Mar 17, 2016

Using the consul example terraform module, it exits prematurely in the first script which is run:

#!/bin/bash
set -xeuo pipefail

sleep 10

# Read the address to join from the file we provisioned
JOIN_ADDRS=$(cat /tmp/consul-server-addr | tr -d '\n')

sudo apt-get update -y < /dev/null || true
sudo apt-get -y --force-yes install unzip < /dev/null || true 
...

The machine I'm provisioning is a consul node. The terraform module is basically the example one from the terraform project.

It began exiting after the apt-get update or apt-get install, hence the /dev/null and || true, but then after adding the sleep command to the script it began exiting right after or during that. Wat.

Terraform says the provision worked. This doesn't happen in v0.6.12.
It is the first provision script to run.

I also noticed that there were some processes still running after the "failed" provisions:

donal    14588     1  0 21:05 ?        00:00:00 /usr/local/bin/terraform-provisioner-file
donal    14600     1  0 21:05 ?        00:00:00 /usr/local/bin/terraform-provisioner-local-exec
donal    14608     1  0 21:05 ?        00:00:00 /usr/local/bin/terraform-provider-aws
donal    14616     1  0 21:05 ?        00:00:00 /usr/local/bin/terraform-provisioner-remote-exec
@abosley
Copy link

abosley commented Mar 17, 2016

+1
My remote-exec script command exits the first bash script prematurely. Second script if present will run correctly. It appears to be exiting after my check for cloud init being complete is finished.

timeout 180 /bin/bash -c \
  'until stat /var/lib/cloud/instance/boot-finished 2>/dev/null; do echo waiting ...; sleep 1; done'

The output from this is shown, then returns 0. I have more code after this:

sudo apt-get -y update
sudo apt-get -y install debconf-utils iperf

echo 'openswan openswan/install_x509_certificate select false
      openswan openswan/restart select true
      openswan openswan/runlevel_changes note
      openswan openswan/x509_self_signed select false' | sudo debconf-set-selections
sudo DEBIAN_FRONTEND=noninteractive apt-get --yes --force-yes install openswan

It never executes according to the output from the apply operation

@gravaton
Copy link

I'm seeing similar behavior within an OpenStack environment - scripts are exiting prematurely or occasionally (as far as I can tell) not being invoked at all. "sleep" seems to exacerbate the issue, but I'm still trying to figure out precisely what's going on. Will update if I get more information.

@byrnedo
Copy link
Author

byrnedo commented Mar 21, 2016

In addition, my atlas_artifact doesn't get picked up either in 0.6.13 but works fine in 0.6.12

@phinze phinze self-assigned this Mar 21, 2016
phinze added a commit that referenced this issue Mar 21, 2016
The script cleanup step added in #5577 was positioned before the
`cmd.Wait()` call to ensure the command completes. This was causing
non-deterministic failures, especially for longer runnings scripts.

Fixes #5699
Fixes #5737
phinze added a commit that referenced this issue Mar 21, 2016
The script cleanup step added in #5577 was positioned before the
`cmd.Wait()` call to ensure the command completes. This was causing
non-deterministic failures, especially for longer running scripts.

Fixes #5699
Fixes #5737
phinze added a commit that referenced this issue Mar 21, 2016
The script cleanup step added in #5577 was positioned before the
`cmd.Wait()` call to ensure the command completes. This was causing
non-deterministic failures, especially for longer running scripts.

Fixes #5699
Fixes #5737
phinze added a commit that referenced this issue Mar 21, 2016
The script cleanup step added in #5577 was positioned before the
`cmd.Wait()` call to ensure the command completes. This was causing
non-deterministic failures, especially for longer running scripts.

Fixes #5699
Fixes #5737
phinze added a commit that referenced this issue Mar 21, 2016
The script cleanup step added in #5577 was positioned before the
`cmd.Wait()` call to ensure the command completes. This was causing
non-deterministic failures, especially for longer running scripts.

Fixes #5699
Fixes #5737
@ghost
Copy link

ghost commented Apr 27, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@ghost ghost locked and limited conversation to collaborators Apr 27, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants