Skip to content

Commit

Permalink
(MODULES-8348) Acceptance scaffold with beaker-puppet
Browse files Browse the repository at this point in the history
Adds an acceptance directory containing a set of scaffold test files
and helpers to allow for testing this module with beaker-puppet instead
of beaker-puppet_install_helper (which is no longer supported). The
README in the acceptance directory describes how to run the new tests.

This leaves the existing tests in spec/acceptance intact for now; in the
future these old tests will either be migrated to the acceptance
directory, or (if they test puppet 3 functionality) removed.

Take note that this process will also eventually remove beaker-rspec,
since it is incompatible with the use of subcommands in beaker 4.
  • Loading branch information
caseywilliams committed Jan 24, 2019
1 parent f0af222 commit 945b8c7
Show file tree
Hide file tree
Showing 10 changed files with 620 additions and 0 deletions.
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
*.iml
.*.sw[op]
.DS_Store
.beaker/
.bundle/
.idea/
.metadata
Expand All @@ -13,6 +14,8 @@ Gemfile.lock
bin/
coverage/
doc/
hosts.yaml
hosts.yml
junit/
log/
pkg/
Expand Down
28 changes: 28 additions & 0 deletions acceptance/Gemfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
source ENV['GEM_SOURCE'] || "https://rubygems.org"

# Find a location or specific version before installing a gem
#
# @param place_or_version can be one of:
# - A specific version,
# - A git branch, as `git://<your-repo>.git#<branch-name>`
# - A file URI, as `file:///absolute/file/path`
def location_for(place, fake_version = nil)
if place =~ /^(git[:@][^#]*)#(.*)/
[fake_version, { git: $1, branch: $2, require: false }].compact
elsif place =~ /^file:\/\/(.*)/
['>= 0', { path: File.expand_path($1), require: false }]
else
[place, { require: false }]
end
end

gem "rake", "~> 12.3"

gem "beaker", *location_for(ENV['BEAKER_VERSION'] || '~> 4')
gem "beaker-puppet", *location_for(ENV['BEAKER_PUPPET_VERSION'] || '~> 1')

gem "beaker-docker", *location_for(ENV['BEAKER_DOCKER_VERSION'] || '~> 0')
gem "beaker-vagrant", *location_for(ENV['BEAKER_VAGRANT_VERSION'] || '~> 0')
gem "beaker-vmpooler", *location_for(ENV['BEAKER_VMPOOLER_VERSION'] || '~> 1')
gem "beaker-hostgenerator", *location_for(ENV['BEAKER_HOSTGENERATOR_VERSION'] || '~> 1')
gem "beaker-abs", *location_for(ENV['BEAKER_ABS_VERSION'] || '~> 0')
125 changes: 125 additions & 0 deletions acceptance/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
# Acceptance tests for puppetlabs-puppet_agent

These integration tests use the [beaker](https://github.com/puppetlabs/beaker)
acceptance test framework to test puppet-agent installation and upgrades with
the puppetlabs-puppet_agent module.

## Quick start

If you are already familiar with beaker, you can get started like this:

```sh
# Install the dependencies
bundle install
# Create a hosts.yml file in this directory with at least one master and one agent
bundle exec beaker-hostgenerator -t docker centos7-64mcda-debian8-64a > hosts.yml
# Use the `prepare` rake task to provision your hosts and set up the master with the latest puppet 5 agent and server:
MASTER_COLLECTION=puppet5 bundle exec rake prepare
# Run the tests
bundle exec beaker exec ./tests/
# Destroy your test hosts
bundle exec beaker destroy
```

See "How to run the tests", below, for more detail.

## Background

### About Beaker

Beaker is a host provisioning and an acceptance testing framework. If you are
unfamiliar with beaker, you can start with these documents:

- [The Beaker DSL document](https://github.com/puppetlabs/beaker/blob/master/docs/how_to/the_beaker_dsl.md) will help you understand the test code in the `tests/` and `pre_suite/` subdirectories.
- [The Beaker Style Guide](https://github.com/puppetlabs/beaker/blob/master/docs/concepts/style_guide.md) will help you write new test code.
- [Argument Processing](https://github.com/puppetlabs/beaker/blob/master/docs/concepts/argument_processing_and_precedence.md) and [Using Subcommands](https://github.com/puppetlabs/beaker/blob/master/docs/tutorials/subcommands.md) have more information on beaker's command line and environmental options.

### About these tests

This module is responsible for upgrading and downgrading puppet-agent. Testing
this behavior necessarily involves repeatedly installing and uninstalling
puppet-agent. Ideally, the test hosts would be totally destroyed and
reprovisioned before each fresh install of puppet-agent, but beaker does not
support workflows like this. Instead, helper methods are used to install
puppet-agent on agent hosts at the beginning of each test and to uninstall it
during teardown. See [helpers.rb](./helpers.rb) for more.

#### Environment variables and the `prepare` rake task

The `prepare` rake task runs `beaker init`, `beaker provision`, and `beaker
pre-suite` all at once to provision your test hosts and prepare you to run
`beaker exec` on the tests you care about.

The pre-suite installs a puppet-agent package and a compatible puppetserver
package on the master host in preparation for running tests on the agent hosts.
It also installs this module (from your local checkout) and its dependencies.

The versions of puppet-agent and puppetserver installed on the master during
the pre-suite can be controlled in two ways:

- set `MASTER_COLLECTION` to 'pc1' (for puppet 4), 'puppet5', or 'puppet6' to
install the latest releases from those streams, or
- set `MASTER_PACKAGE_VERSION` to a specific version of puppet-agent (like
'5.5.10') to install that agent package and a compatible puppetserver

You may also set `DEBUG` to run beaker in debug mode.

## How to run the tests

### Install the dependencies

This directory has its own Gemfile, containing gems required only for these
acceptance tests. Ensure that you have [bundler](https://bundler.io/) installed,
and then use it to install the dependencies:

```sh
bundle install --path .bundle
```

This will install [`beaker`](https://github.com/puppetlabs/beaker) and
[`beaker-puppet`](https://github.com/puppetlabs/beaker-puppet) (a beaker
library for working with puppet specifically), plus several hypervisor gems for
working with beaker and vagrant, docker, or vsphere.

### Set up the test hosts

Use `beaker-hostgenerator` generate a hosts file that describes the types of
hosts you want to test. See beaker-hostgenerator's help for more information on
available host OSes, types and roles.

Make sure your set of test hosts has at least one host with the master role and
one host with the agent role. This example creates a Centos 7 master and a
single Debian 9 agent, which will be provisioned with Docker:

```sh
bundle exec beaker-hostgenerator -t docker centos7-64mcda-debian9-64a > ./hosts.yaml
```

Decide on a collection or version of puppet-agent to use on your master, and
run the `prepare` rake task to set it up. This example installs the latest
puppet-agent and puppetserver in the puppet 5 series on the master:

```sh
MASTER_COLLECTION=puppet5 bundle exec rake prepare
````

### Run and re-run the tests

Once you've set up beaker, you can run any number of tests any number of times:
```sh
# Run all the tests
bundle exec beaker exec ./tests/
# Run all the tests in a specific directory
bundle exec beaker exec ./tests/subdir
# Run a commma-separated list of specific tests:
bundle exec beaker exec ./path/to/test.rb,./another/test.rb
```
### Clean up
To destroy the provisioned test hosts:
```sh
bundle exec beaker destroy
```
60 changes: 60 additions & 0 deletions acceptance/Rakefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
def beaker(command)
debug_flag = ''
if ENV['BEAKER_debug'] || ENV['DEBUG']
debug_flag = '--debug'
end

sh(%(beaker #{command} #{debug_flag}))
end

desc 'Prepare for running tests: Run beaker init, provision, and pre-suite all at once.'
task :prepare do
errors = []

if !ENV['MASTER_PACKAGE_VERSION'] && !ENV['MASTER_COLLECTION']
errors << 'You must set a starting version of puppet-agent using ' +
'$MASTER_PACKAGE_VERSION or $MASTER_COLLECTION'
end

if ENV['HOSTS']
hosts_file = File.expand_path(ENV['HOSTS'])
errors << "Couldn't find $HOSTS file at #{hosts_file}" unless File.exist?(hosts_file)
elsif File.exist?('hosts.yml')
hosts_file = './hosts.yml'
elsif File.exist?('hosts.yaml')
hosts_file = './hosts.yaml'
else
errors << 'Unable to find a hosts file in $HOSTS or at ./hosts.yml or at ./hosts.yaml'
end

unless errors.empty?
raise errors.join("\n")
end

puts "Using beaker hosts file #{hosts_file}"

# Log the contents of the host file if running in Jenkins
if ENV['JENKINS_HOME']
pp File.read(hosts_file)
end

beaker("init -h #{hosts_file} -o options.rb")
beaker("provision")
beaker("exec ./pre_suite")

# Don't print these human instructions if running in Jenkins
unless ENV['JENKINS_HOME']
puts 'You can run individual test(s) with `bundle exec beaker exec <path-to-test(s)>`'
puts "You can destroy your provisioned hosts with `bundle exec beaker destroy` when you're ready"
end
end

desc 'Run all the tests and destroy the hosts afterward'
task :ci do
begin
Rake.application['prepare'].invoke
beaker('exec ./tests')
ensure
beaker('destroy')
end
end
13 changes: 13 additions & 0 deletions acceptance/files/uninstall.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
$agentVer = Get-ChildItem -Path HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall, HKLM:\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall |
Get-ItemProperty |
Where-Object {$_.DisplayName -like "Puppet Agent*" } |
Select-Object -Property DisplayName, UninstallString

ForEach ($ver in $agentVer) {
If ($ver.UninstallString) {
$uninst = "start /wait "+$ver.UninstallString+" /quiet /norestart /l*vx uninstall_puppet.log"
Write-Host "Uninstalling: $uninst"
& cmd.exe /c $uninst
}

}
Loading

0 comments on commit 945b8c7

Please sign in to comment.