Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Busser and chef-client-zero.rb Gem Sandboxing #258

Merged
merged 63 commits into from
Nov 26, 2013

Conversation

fnichol
Copy link
Contributor

@fnichol fnichol commented Nov 20, 2013

References #225, #240, #241, and #242

This reverts commit 16fb8c2.

Conflicts:
	lib/kitchen/provisioner/chef_zero.rb
This reverts commit 291c1db.

Conflicts:
	lib/kitchen/provisioner/chef_zero.rb
The directories that will be rejected when search for suite files
include:

* data/
* data_bags/
* environments/
* nodes/
* roles/
To override the directory containing the `ruby` and `gem` commands
(which defaults to `/opt/chef/embedded/bin`), simply set the following
in a `driver_config` block in a kitchen.yml file:

    ---
    driver_config:
      instance_ruby_bindir: /opt/local/ruby/bin

Busser will use this path to calculate the path to `ruby`, `gem`, and
the default `GEM_PATH` environment.

Note this will be the value as reported by:

    RbConfig::CONFIG["bindir"]

References 291c1db
Delegate the environment variable maintenance back to busser except when
initially installing the gem and running `busser setup`. This will
generate a `$BUSSER_ROOT/bin/busser` binstub that isolates its gem
environment, path to `ruby`, etc.

When calculating Busser's GEM_HOME, it will be set to
`$BUSSER_ROOT/gems`. The GEM_PATH will be calculated by prepending the
value of GEM_HOME onto the existing GEM_PATH.

This means that if the Ruby being used is an Omnibus package then the
Chef gem will be LOAD_PATH require'able but any gem installation made by
Busser will go under its BUSSER_ROOT. This includes the initial Busser
gem installation.

References #242
References #225
@ghost ghost assigned fnichol Nov 20, 2013
@eklein
Copy link

eklein commented Nov 21, 2013

@petecheslock @fnichol this fixed it!

before (on beta4):

   Failures:

     1) java_service_test::default Command "/opt/chef/embedded/bin/bluepill echoserver status" should return stdout /up/
        Failure/Error: it { should return_stdout /up/ }
   /opt/chef/embedded/bin/bluepill echoserver status
   /opt/chef/embedded/lib/ruby/site_ruby/1.9.1/rubygems/dependency.rb:247:in `to_specs': Could not find bluepill (>= 0) amongst [busser-0.5.0, busser-serverspec-0.2.5, diff-lcs-1.2.5, highline-1.6.20, net-ssh-2.7.0, rspec-2.14.1, rspec-core-2.14.7, rspec-expectations-2.14.4, rspec-mocks-2.14.4, serverspec-0.11.4, thor-0.18.1] (Gem::LoadError)
    from /opt/chef/embedded/lib/ruby/site_ruby/1.9.1/rubygems/dependency.rb:256:in `to_spec'
    from /opt/chef/embedded/lib/ruby/site_ruby/1.9.1/rubygems.rb:1231:in `gem'
    from /opt/chef/embedded/bin/bluepill:22:in `<main>'

after (on branch):

   Finished in 0.48729 seconds
   10 examples, 0 failures

awesome!

I believe this is the exact same use-case that @petecheslock had, so hopefully that is enough of a positive sign. Let me know if you want me to do more investigating.

@fnichol
Copy link
Contributor Author

fnichol commented Nov 21, 2013

@eklein Sweet, thanks for the confirmation! It looks like Chef 11.8.0 introduced RSpec into the Omnibus package so this Serverspec may still have an issue loading but I have an idea for a solution to that.

@eklein
Copy link

eklein commented Nov 21, 2013

@fnichol, that's using serverspec and chef-zero above.. ugh, just noticed it's using chef 11.6. Let me give it a shot with 11.8.

@fnichol
Copy link
Contributor Author

fnichol commented Nov 21, 2013

@eklein No problem, you've confirmed what I've seen as well--this is good, we're getting there!

Currently the following values are supported:

* `"busser"`, `"some-name"`: alternative name for gem
* `"1.2.3"`, `"0.7.0.beta.2"`: specific RubyGems version
* `"busser@0.5.0"`, `"some-name@5.0.0"`: specific gem name and version
* `"/tmp/busser-0.6.0.dev.gem"`: path on remote instance to gem pkg
Previous to this commit, Test Kitchen's work directory on the instance
depended on the provisioner used. For example the chef_solo provisioner
worked out of `/tmp/kitchen-chef-solo` and the chef_zero provisioner
worked out of `/tmp/kitchen-chef-zero`.

With the addition of a `data_path` config option this made discovering
the path to uploaded data more difficult. Now all provisioners will
upload the contents of `data_path` to `/tmp/kitchen/data`.
Shorthand:

    ---
    driver: dummy

Explicit:

    ---
    driver:
      name: dummy
      other: stuff
Due to the way multi-YAML file merging happens, there may end up being a
data structure that contains `:driver`, `:driver_plugin`, and
`driver_config` all in the same location (i.e. in a platform, suite, or
common).

In case of current `:driver` and legacy keys, the newer `:driver` config
will win in the recursive merge and take precedence over the legacy
format.
This effectively will convert the following:

    ---
    suites:
    - name: foo
      data_path: /my/data
      data_bags_path: /my/data_bags
      environments_path: /my/environments
      nodes_path: /my/nodes
      roles_path: /my/roles

into:

    ---
    suites:
    - name: foo
      provisioner:
        data_path: /my/data
        data_bags_path: /my/data_bags
        environments_path: /my/environments
        nodes_path: /my/nodes
        roles_path: /my/roles

This will be supported in the near-term for backwards compatability,
then a future release will deprecate this legacy usage with a warning
when encountered, then the legacy support will be finally removed in
another release.
Introduce default config, computed defaults for various Chef data
paths, compute solo.rb and client.rb from a Hash so that settings can be
overridden or added to.
This fully isolates Busser and its runner plugins to a GEM_HOME and
GEM_PATH of `<busser_root_path>/gems`.
This may return in another form in the future, promise!
This preserves prior merging behavior of run_list arrarys:

1. items from a run_list on the platform will be placed first
2. items from a run_list on the suite will be placed next
To make this work, a modern version of Chef will be installed into a
sandboxed path (no dependencies at all on Omnibus packages gems) and
used to start chef-zero on a thread during a normal chef-client run
(using your desired Chef version).

This is best-effort for versions of Chef less than 11.8.0, there may
be missing package dependencies needed on your specific platform.
If this is the case, the chef_zero Provisioner may crash and fail
the converge action.

Note that versions of Chef greater than or equal to 11.8.0 will be
supported out of the box with Test Kitchen running chef-client in
local mode to get the desired functionality.
This can be useful in `.kitchen/logs/kitchen.log` when tracking dates,
times and durations of `kitchen` command invocations.
Will be investigating Rubocop as an alternative.
fnichol added a commit that referenced this pull request Nov 26, 2013
Fix Busser and chef-client-zero.rb Gem Sandboxing
@fnichol fnichol merged commit eea1e53 into master Nov 26, 2013
@jtimberman jtimberman deleted the fix-busser-sandboxing branch November 26, 2013 20:04
BrentOnRails pushed a commit to BrentOnRails/test-kitchen that referenced this pull request Jul 17, 2017
This allows a user to rely on a [default] profile getting picked up in
the shared credentials file (~/.aws/credentials).

Fixes test-kitchen#295
Fixes test-kitchen#258
@test-kitchen test-kitchen locked and limited conversation to collaborators Nov 16, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants