Plugin install isn't idempotent

[spuder]

I find that installing some plugins doesn't work on subsequent chef runs. It gives the error

ERROR: plugin directory /usr/share/elasticsearch/plugins/hq already exists.

           Cookbook Trace:
           ---------------
           /tmp/kitchen/cookbooks/elasticsearch/libraries/provider_plugin.rb:44:in `block (2 levels) in <class:PluginProvider>'
           /tmp/kitchen/cookbooks/elasticsearch/libraries/provider_plugin.rb:27:in `block in <class:PluginProvider>'

           Resource Declaration:
           ---------------------
           # In /tmp/kitchen/cookbooks/nd-elasticsearch/recipes/default.rb

            70: elasticsearch_plugin 'royrusso/elasticsearch-HQ' do
            71:   action :install
            72: end

           Compiled Resource:
           ------------------
           # Declared in /tmp/kitchen/cookbooks/nd-elasticsearch/recipes/default.rb:70:in `from_file'

           elasticsearch_plugin("royrusso/elasticsearch-HQ") do
             action [:install]
             retries 0
             retry_delay 2
             default_guard_interpreter :default
             declared_type :elasticsearch_plugin
             cookbook_name :"nd-elasticsearch"
             recipe_name "default"
             plugin_name "royrusso/elasticsearch-HQ"
           end


       Running handlers:
       [2015-11-21T06:06:41+00:00] ERROR: Running exception handlers
       Running handlers complete
       [2015-11-21T06:06:41+00:00] ERROR: Exception handlers complete
       Chef Client failed. 7 resources updated in 01 minutes 08 seconds
       [2015-11-21T06:06:41+00:00] FATAL: Stacktrace dumped to /tmp/kitchen/cache/chef-stacktrace.out
       [2015-11-21T06:06:41+00:00] ERROR: elasticsearch_plugin[royrusso/elasticsearch-HQ] (nd-elasticsearch::default line 70) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '74'
       ---- Begin output of ["/usr/share/elasticsearch/bin/plugin", "install", "royrusso/elasticsearch-HQ"] ----
       STDOUT: -> Installing royrusso/elasticsearch-HQ...
       Trying https://github.com/royrusso/elasticsearch-HQ/archive/master.zip ...
       Downloading ...................................................................................................................................................................DONE
       Verifying https://github.com/royrusso/elasticsearch-HQ/archive/master.zip checksums if available ...
       NOTE: Unable to verify checksum for downloaded plugin (unable to find .sha1 or .md5 file to verify)
       ERROR: plugin directory /usr/share/elasticsearch/plugins/hq already exists. To update the plugin, uninstall it first using 'remove hq' command
       STDERR:
       ---- End output of ["/usr/share/elasticsearch/bin/plugin", "install", "royrusso/elasticsearch-HQ"] ----
       Ran ["/usr/share/elasticsearch/bin/plugin", "install", "royrusso/elasticsearch-HQ"] returned 74
       [2015-11-21T06:06:41+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
>>>>>> Converge failed on instance <default-ubuntu-1404>.
>>>>>> Please see .kitchen/logs/default-ubuntu-1404.log for more details
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: SSH exited (1) for command: [sh -c '

sudo -E /opt/chef/bin/chef-solo --config /tmp/kitchen/solo.rb --log_level auto --force-formatter --no-color --json-attributes /tmp/kitchen/dna.json
']
>>>>>> ----------------------

Strangely it doesn't manifest itself with every plugin.

royrusso/elasticsearch-HQ Always fails the second converge however lmenezes/elasticsearch-kopf and mobz/elasticsearch-head succeed.

I can't find any documentation showing why HQ might not work. My theory is that maybe it doesn't support ES 2.0.0

elasticsearch_plugin 'mobz/elasticsearch-head' do
  action :install
end

# Paramedic currently doesn't want to install on es 2.0.0. I emailed creator since no github issues enabled.
# elasticsearch_plugin 'karmi/elasticsearch-paramedic' do
#   action :install
# end

elasticsearch_plugin 'lmenezes/elasticsearch-kopf' do
  action :install
end

elasticsearch_plugin 'royrusso/elasticsearch-HQ' do
   action :install
 end

[martinb3]

As mentioned in #264, there's no API or method for detecting if a plugin can be upgraded (or even if a particular version is installed or compatible to be installed). This seems to be a limitation of Elasticsearch plugins, so I am not sure what to offer as a solution. I think all we can do at the chef level is provide a way to call bin/plugin in ES with all possible options.

Regarding the "HQ" plugin in particular, it seems that /usr/share/elasticsearch/plugins/hq is the directory being used, which I've never seen before -- in every other instance, the directory name is the same as the plugin name. This seems like it's probably just a bad plugin.

I'm happy to document this issue further, though, so other folks don't run into it.

[martinb3]

I'm going to rework the plugin resource a bit to make this case easier. Please stand by.

[spuder]

On the latest 2.0.0_wip branch, adding the name makes the HQ plugin idempotent.

elasticsearch_plugin 'royrusso/elasticsearch-HQ' do
  name 'hq'
  action :install
end

Update

I find that using just the URL fails

           STDERR:
           ---- End output of ["/usr/share/elasticsearch/bin/plugin", "install", "royrusso/elasticsearch-HQ"] ----
           Ran ["/usr/share/elasticsearch/bin/plugin", "install", "royrusso/elasticsearch-HQ"] returned 74

           Cookbook Trace:
           ---------------
           /tmp/kitchen/cookbooks/elasticsearch/libraries/provider_plugin.rb:38:in `block in <class:PluginProvider>'

           Resource Declaration:
           ---------------------
           # In /tmp/kitchen/cookbooks/nd-elasticsearch/recipes/default.rb

            65: elasticsearch_plugin 'royrusso/elasticsearch-HQ' do
            66:   url 'royrusso/elasticsearch-HQ'
            67:   # name 'hq' #https://github.com/elastic/cookbook-elasticsearch/issues/392
            68:   action :install
            69: end

           Compiled Resource:

           # Declared in /tmp/kitchen/cookbooks/nd-elasticsearch/recipes/default.rb:65:in `from_file'

           elasticsearch_plugin("royrusso/elasticsearch-HQ") do
             action [:install]
             retries 0
             retry_delay 2
             default_guard_interpreter :default
             declared_type :elasticsearch_plugin
             cookbook_name :"nd-elasticsearch"
             recipe_name "default"
             url "royrusso/elasticsearch-HQ"
             plugin_name "royrusso/elasticsearch-HQ"
           end


       Running handlers:
       [2015-11-22T01:11:09+00:00] ERROR: Running exception handlers
       Running handlers complete
       [2015-11-22T01:11:09+00:00] ERROR: Exception handlers complete
       Chef Client failed. 1 resources updated in 21 seconds
       [2015-11-22T01:11:09+00:00] FATAL: Stacktrace dumped to /tmp/kitchen/cache/chef-stacktrace.out
       [2015-11-22T01:11:09+00:00] ERROR: elasticsearch_plugin[royrusso/elasticsearch-HQ] (nd-elasticsearch::default line 65) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '74'
       ---- Begin output of ["/usr/share/elasticsearch/bin/plugin", "install", "royrusso/elasticsearch-HQ"] ----
       STDOUT: -> Installing royrusso/elasticsearch-HQ...
       Trying https://github.com/royrusso/elasticsearch-HQ/archive/master.zip ...
       Downloading ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................DONE
       Verifying https://github.com/royrusso/elasticsearch-HQ/archive/master.zip checksums if available ...
       NOTE: Unable to verify checksum for downloaded plugin (unable to find .sha1 or .md5 file to verify)
       ERROR: plugin directory /usr/share/elasticsearch/plugins/hq already exists. To update the plugin, uninstall it first using 'remove hq' command
       STDERR:
       ---- End output of ["/usr/share/elasticsearch/bin/plugin", "install", "royrusso/elasticsearch-HQ"] ----
       Ran ["/usr/share/elasticsearch/bin/plugin", "install", "royrusso/elasticsearch-HQ"] returned 74
       [2015-11-22T01:11:10+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
>>>>>> Converge failed on instance <default-ubuntu-1404>.
>>>>>> Please see .kitchen/logs/default-ubuntu-1404.log for more details
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: SSH exited (1) for command: [sh -c '

sudo -E /opt/chef/bin/chef-solo --config /tmp/kitchen/solo.rb --log_level auto --force-formatter --no-color --json-attributes /tmp/kitchen/dna.json
']
>>>>>> ----------------------

[martinb3]

In all of the README examples, the resource name is simply the plugin name now, e.g.:

elasticsearch_plugin 'hq' do
  url 'royrusso/elasticsearch-HQ'
  # name 'hq' #https://github.com/elastic/cookbook-elasticsearch/issues/392
  action :install
end

royrusso/elasticsearch-HQ isn't the actual valid plugin name -- it's hq.