Gem path issues after test-kitchen beta 4 new sandbox. #242
Comments
ghost
assigned 
|
@petecheslock still thinking about the best way to fix this, but as a workaround you could add this to your ENV['GEM_HOME'] = nil
ENV['GEM_PATH'] = nil
ENV['GEM_CACHE'] = nil |
|
Ahh sweet - that workaround works. Thanks - a more perm fix will be nice - but this is a huge help. |
|
Any movement on this? We need some of the fixes that are coming in beta5 (and that were in beta4 with the other stuff that broke) and I've been told this issue is what's blocking that release :) Anything I can help with? |
|
Sorry all, I am working on this (last night and today). I'm having to rejigger the codebase in a few places to make Test Kitchen dumber and Busser a bit smarter as it relates to its gem environment.
|
|
@fnichol certainly, just let me know once it's ready to go and if needed, which branch it's on.. |
|
@fnichol roger that, just @ mention me on anything you need reviewed. |
|
Totally. Give me a @ mention and I'll be super happy to help out.
|
Delegate the environment variable maintenance back to busser except when initially installing the gem and running `busser setup`. This will generate a `$BUSSER_ROOT/bin/busser` binstub that isolates its gem environment, path to `ruby`, etc. When calculating Busser's GEM_HOME, it will be set to `$BUSSER_ROOT/gems`. The GEM_PATH will be calculated by prepending the value of GEM_HOME onto the existing GEM_PATH. This means that if the Ruby being used is an Omnibus package then the Chef gem will be LOAD_PATH require'able but any gem installation made by Busser will go under its BUSSER_ROOT. This includes the initial Busser gem installation. References #242 References #225
|
Ok, we're getting there. There are a couple of More to follow, but if @petecheslock and @eklein are up for trying an in-progress branch then that's cool by me 😄 |
|
@petecheslock think you could link or gist that serverspec test? |
|
@fnichol, i'll take this for a spin in an hour or so. |
|
i can also duplicate the serverspec test that @petecheslock attempted above if that helps. |
|
Actually - i'm blocked on testing this due to #255 Not sure if there is a hack/patch I can do to get around this. |
|
@petecheslock understood, I shelved that issue for this one. If I can resolve it fast I'll revert the feature for now |
|
Ok - here where i'm at. I checked out this branch - did a rake install in there (after a quick and dirty hack for #255) I removed my workaround for this from @schisamo (this was in my serverspec spec_helper.rb) Then did a kitchen verify https://gist.github.com/petecheslock/227c45cb749b6b9ad494 That also includes the test i'm using (a simple one that checks some crons and chef versions) More of a POC for us internally. |
|
Oh - actually that gem issue happens no matter what. Here are my local ruby/gem details: https://gist.github.com/petecheslock/2557d4d9c177843ebe46 |
|
Here is another test using serverspec ~> 0.11 https://gist.github.com/petecheslock/9f6077e6bac37b740032 |
|
what's in your gemfile.lock for rspec/rspec-core? |
|
it looks like somehow your bundle is trying to use rspec 2.13.x with rspec-core 2.14.x.. what if you bundle update? we're using rspec 2.14.1 and rspec-core 2.14.7 without issues.. |
|
Let me give that a shot. |
|
So a couple of things that should shed some light on the issues @petecheslock is seeing in his gists above. Busser doesn't use the Gemfile you have in your cookbook project. It does a gem install for each plugin (based on directory naming If you are using Test Kitchen's previous non-sandbox behavior then Busser would install into Omnibus Chef's gem path. This can cause some issues with some versions of the Omnibus Chef 11.8.0 as they ship with RSpec: This is mainly to aid in the easy execution of functional tests in our CI pipelines.The issue is the version of RSpec that ships in Omnibus Chef conflicts with the version of RSpec serverspec wants to load: https://github.com/opscode/chef/blob/master/chef.gemspec#L45 Gem::Specification.new do |s|
s.name = 'chef'
...
%w(rspec-core rspec-expectations rspec-mocks).each { |gem| s.add_development_dependency gem, "~> 2.13.0" }
...
endhttps://github.com/serverspec/serverspec/blob/master/serverspec.gemspec#L22 Gem::Specification.new do |spec|
spec.name = "serverspec"
...
spec.add_runtime_dependency "rspec", ">= 2.13.0"
...
endThe new sandboxing should fix this though as Busser uses a completely isolated Gem install. One reason this sandboxing is important is we (Opscode) don't make any gunrantees of what will happen when you install gems into the Omnibus Chef gem path. As you can imagine supporting this can prove troublesome. I'll let @fnichol chime in with any additions to my statements above. |
|
@schisamo There's one problem though: Busser depends on Chef for it's Guess I'll need to survey the busser plugins to see what the damage might be of severing the Chef load dependency. Thoughts? |
|
@fnichol Yeah for now I think's it's best to completely remove Busser's indirect Chef dependency . Busser's chef_apply helper is neat (and useful) but until we refactor some bits of Chef into a library, dep conflicts like this may prove troublesome.
Agreed, I'll work with the Chef core team to see if there is a way we can isolate Chef's development dependencies in the Omnibus install. /cc @danielsdeleo @sersut |
|
After looking at busser runner plugins in the wild (GitHub and RubyGems) there's only one that I can find using the I think we could PR that away which leaves one other (hopefully) under or unused feature which had a code dependency on Chef. If so, then Busser becomes slightly more portable :) |
|
I'm still convinced there must be something more to this problem. Chef has had a development dependency on a particular version of rspec since before tk existed. Looking at the rubygems code, there's no way development dependencies should matter when resolving the runtime dependencies. And I can't cause the conflict manually: As for chef including its test code in the omnibus package, I think we've tried every permutation of not including test code in the package, and had problems with each way of doing this. To ensure that we're testing the package and not build artifacts that might be leftover, we run tests on different machines than we built the packages with. In the past, we installed the test code on the test machines only, but frequently we would have problems with the packages that would be hidden by bundler installing additional gems. So we learned that trying to install new software on the test system can hide bugs and render our tests meaningless. |
|
@danielsdeleo I actually enjoy the fact that testing code is in there, self-verifying code is awesome! I think the problems above were a gem activation/load ordering issue where a newer 'rspec' gem was loaded and found and older 'rspec-core' already activated. Ultimately the best solution in the long term for Busser is to pretend like the Omnibus gems don't exist, but optimistically use its Ruby installation if available. Y'know, totally needy but acting all independent-like. |
|
👍 If there's any way I can help on the Chef Core side of things, don't hesitate to ask. |
|
@petecheslock my apologies if I sent you on a wild goose chase.. I attempted running with chef 11.8.0 and now am hitting the same issue as you. I hadn't realized that I was running on 11.6.0 before.. |
|
@fnichol any word? |
|
@eklein sorry for the delay getting back to you, rest assured I've been very with some refactoring which will help in the long term… |
|
While the gem sandboxing will be fixed for good when #258 is merged, there is still an underlying issue with serverspec specifically (not the busser-serverspec runner plugin). It turns out that @petecheslock has the mother-of-all good failure cases: when invoking a Ruby program that inherits For Busser to isolate its work all gems (dependencies, runner plugins, and runner plugin dependencies) are now installed into So Busser makes sure that The problem with running It looks to me what we need is a way to configure/tune/adjust/modify Serverspec's Exec Backend to ensure that the following environment variables are not present when fork/exec'ing:
That way when So to recap, setting I've started a pull request on mizzy/serverspec#300 to address this and would appreciate help anyone can provide testing, verifying, etc. |
|
For anyone following along at home, here's what I used to test the above Serverspec PR with (in require 'serverspec'
require 'pathname'
include Serverspec::Helper::Exec
include Serverspec::Helper::DetectOS
RSpec.configure do |c|
c.before :all do
c.os = backend(Serverspec::Commands::Base).check_os
c.path = '/sbin:/usr/sbin'
end
end
describe "Chefness" do
it "should be the right version of Chef" do
expect(command "chef-client --version").to return_stdout("Chef: 11.8.0")
end
it "should have not have GEM_HOME set" do
expect(command "echo $GEM_HOME").to return_stdout("")
end
endNote that I logged into the instance (using |
|
@fnichol WOW - Seriously impressive detective work. I'm glad to know that if there is one thing i'm good at - it's developing solid ways to fail \o/ I see that mizzy/serverspec#300 got merged into v0.11.5 so i'm going to check this all out and test. |
|
WOOHOO! Running HEAD on test-kitchen and serverspec v0.11.5 these tests all pass for me now. No longer seeing any of the issues around this from before. How do we feel about cutting a new beta release tag and push to rubygems - bunch of fixes are available and this would help to get more people on my team to easily pull in those updates (without needing to build from HEAD) |
|
woohoo indeed. I'd love to get this out there too. Please let me know if I can help in any way. |
|
YESSSS!! I'll do you one better, I'm planning to cut 1.0.0.rc.1 tomorrow. Shake it down a little and then… |
|
|
|
Also - THANK YOU! |
|
hahaha that picture. agreed on a big thank you. 👍 |
|
We did it, thank you all! |
|
Ok - just one more 'cause Dave Grohl |
Using ServerSpec to test for local installed versions of either gems or binaries installed via gem. (in this case chef-client).
The serverspec test
or this test
in TK beta.3
In TK >= beta.4
The 2nd test
The text was updated successfully, but these errors were encountered: