Terraform action hanging forever on prompt for sudo on step Cleanup user key...

[eyalzek]

As part of the new TF release (0.7.4) the option to recreate a chef client was added (recreate_client = true). I had to tweak the chef provisioner settings a bit, replacing validation_client_name & validation_key with user_name & user_key.
After creating and provisioning a node, terraform attempts to cleanup the user keys it used during the first run. The command used for that is ran as the ubuntu user with sudo.
The problem is, that as a part of our provisioning, we remove the ubuntu user from sudoers, causing a prompt for sudo password, which will never time out and never finish. Here is some output from both a failure and a successful run:
Failure:

module.env.modulename.aws_instance.server.0 (chef): Running handlers:
module.env.modulename.aws_instance.server.0 (chef): [2016-09-20T09:37:38+00:00] ERROR: Running exception handlers
module.env.modulename.aws_instance.server.0 (chef): Running handlers complete
module.env.modulename.aws_instance.server.0 (chef): [2016-09-20T09:37:38+00:00] ERROR: Exception handlers complete
module.env.modulename.aws_instance.server.0 (chef): Chef Client failed. 163 resources updated in 05 minutes 07 seconds
module.env.modulename.aws_instance.server.0 (chef): [2016-09-20T09:37:38+00:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
module.env.modulename.aws_instance.server.0 (chef): [2016-09-20T09:37:38+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
module.env.modulename.aws_instance.server.0 (chef): [2016-09-20T09:37:38+00:00] ERROR: Chef::Exceptions::MultipleFailures
module.env.modulename.aws_instance.server.0 (chef): [2016-09-20T09:37:38+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
module.env.modulename.aws_instance.server.0 (chef): Cleanup user key...
module.env.modulename.aws_instance.server.0 (chef): [sudo] password for ubuntu:
module.env.modulename.aws_instance.server.0: Still creating... (6m40s elapsed)
module.env.modulename.aws_instance.server.0: Still creating... (6m50s elapsed)
....
module.env.modulename.aws_instance.server.0: Still creating... (10m10s elapsed)
module.env.modulename.aws_instance.server.0: Still creating... (10m20s elapsed)
.............

Successful run:

module.env.modulename.aws_instance.server.0 (chef): Chef Client finished, 239/372 resources updated in 06 minutes 12 seconds
module.env.modulename.aws_instance.server.0 (chef): Cleanup user key...
module.env.modulename.aws_instance.server.0 (chef): [sudo] password for ubuntu:
module.env.modulename.aws_instance.server.0: Still creating... (7m20s elapsed)
module.env.modulename.aws_instance.server.0: Still creating... (7m30s elapsed)
...........................
module.env.modulename.aws_instance.server.0: Still creating... (1h26m41s elapsed)
module.env.modulename.aws_instance.server.0: Still creating... (1h26m51s elapsed)
module.env.modulename.aws_instance.server.0: Still creating... (1h27m1s elapsed)
^CTwo interrupts received. Exiting immediately. Note that data
loss may have occurred.

Terraform Version

0.7.4

Is there a way around it using terraform while keeping our current chef logic? In the meantime I must keep using version 0.7.3.

[cblecker]

You can modify the username used to connect for the chef provisioner by adding the provisioner connections stanza to the chef provisioner block:
https://www.terraform.io/docs/provisioners/connection.html

Using a username other than the ubuntu default should fix this.

[eyalzek]

The default for connection is root is it not?
Anyway I don't think I can connect to an AWS ubuntu instance with any username other than ubuntu when it is just initialized, or am I wrong? So basically, removing the ubuntu user from sudoers becomes impossible when using a chef user instead of validator in the current implementation of terraform..

[svanharmelen]

Hmm... I will have to dive into that one and see if we can work around that... Will come back on this one shortly.

[svanharmelen]

@eyalzek I have a fix for this in #9114 which will be merged shortly...

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.