Inconsistent Apt Cache in a Single-VM Scenario

[tknerr]

I can reproducably make the Chef provisioning of this sample cookbook fail just by enabling cachier:
https://github.com/tknerr/sample-toplevel-cookbook

The Chef run fails with that output:

....
==> sample-app: [2014-07-16T20:20:14+00:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
==> sample-app: [2014-07-16T20:20:14+00:00] ERROR: package[update-notifier-common] (apt::default line 68) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '100'
==> sample-app: ---- Begin output of apt-get -q -y install update-notifier-common=0.119ubuntu8.6 ----
==> sample-app: STDOUT: Reading package lists...
==> sample-app: Building dependency tree...
==> sample-app: Reading state information...
==> sample-app: Suggested packages:
==> sample-app:   gksu
==> sample-app: The following NEW packages will be installed:
==> sample-app:   update-notifier-common
==> sample-app: 0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded.
==> sample-app: Need to get 222 kB of archives.
==> sample-app: After this operation, 1,991 kB of additional disk space will be used.
==> sample-app: Err http://us.archive.ubuntu.com/ubuntu/ precise-updates/main update-notifier-common all 0.119ubuntu8.6
==> sample-app:   404  Not Found [IP: 91.189.91.15 80]
==> sample-app: STDERR: Failed to fetch http://us.archive.ubuntu.com/ubuntu/pool/main/u/update-notifier/update-notifier-common_0.119ubuntu8.6_all.deb  404
  Not Found [IP: 91.189.91.15 80]
==> sample-app: E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
==> sample-app: ---- End output of apt-get -q -y install update-notifier-common=0.119ubuntu8.6 ----
==> sample-app: Ran apt-get -q -y install update-notifier-common=0.119ubuntu8.6 returned 100
==> sample-app: [2014-07-16T20:20:14+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
Chef never successfully completed! Any errors should be visible in the
output above. Please fix your recipes so that they properly complete.

This happens both with either :box and :machine scope. There was no parallel provisioning either.

Steps to reproduce:

• git clone https://github.com/tknerr/sample-toplevel-cookbook
• cd sample-toplevel-cookbook
• vagrant up => works
• edit Vagrantfile and add config.cache.scope = :box
• vagrant destroy -f
• vagrant up => Chef run fails with above output

This happens on Win7 (64bit), Vagrant 1.6.3, vagrant-cachier 0.7.2 (+ vagrant-omnibus 1.4.1, vagrant-berkshelf 2.0.1)

Any ideas?

[tknerr]

No clue what's going on, but what I can say is:

• it happens reproducibly, even on the first vagrant up after ´rm -rf ~/.vagrant.d/cache`
• the http://us.archive.ubuntu.com/ubuntu/pool/main/u/update-notifier/update-notifier-common_0.119ubuntu8.6_all.deb it tries to download returns 404
• the http://us.archive.ubuntu.com/ubuntu/pool/main/u/update-notifier/update-notifier-common_0.119ubuntu8.7_all.deb which it downloads without vagrant-cachier exists

Might it be some HTTP redirect which vagrant-cachier does not follow?

[tknerr]

I guess the root cause for this is #106

After disabling the :apt_lists bucket the Chef run is successful again. Cleaning the cache and enabling :apt_lists causes the Chef run to reliably fail.

[tknerr]

As a temporary workaround to disable the :apt_lists bucket I put this at the top of my global ~/.vagrant.d/Vagrantfile:

# monkey-patch / disable the :apt_lists bucket until this is fixed:
# https://github.com/fgrehm/vagrant-cachier/issues/113
module VagrantPlugins
  module Cachier
    class Bucket
      class AptLists < Bucket
        def self.capability
          :none
        end
      end
    end
  end
end

Vagrant.configure("2") do |config|

  # enable cachier globally
  if Vagrant.has_plugin?("vagrant-cachier")
    config.cache.scope = :box
  end
end

/cc @chrisvire @snowch

[snowch]

@tknerr - thanks for this! Does this completely disable apt caching, or does it just disable one feature of apt caching (apt_lists)?

[tknerr]

@snowch only :apt_lists, the apt packages are still being cached (via the :apt bucket)

[tknerr]

P.S.: the monkeypatch should be better inside the Vagrant.has_plugin guard btw:

# global Vagrant configuration
Vagrant.configure("2") do |config|

  # enable cachier globally
  if Vagrant.has_plugin?("vagrant-cachier")
    # monkey-patch / disable the :apt_lists bucket until this is fixed:
    # https://github.com/fgrehm/vagrant-cachier/issues/113
    module VagrantPlugins
      module Cachier
        class Bucket
          class AptLists < Bucket
            def self.capability
              :none
            end
          end
        end
      end
    end
    config.cache.scope = :box
  end
end

[snowch]

Awesome! Thank you!

[fgrehm]

If the root cause is the windows host, I think we can keep this closed and keep track of a possible fix on GH-106

@tknerr thanks for all the info and tracking it down!