diff --git a/doc/puppetfile.mkd b/doc/puppetfile.mkd index 79d852ce6..6eb1f2da2 100644 --- a/doc/puppetfile.mkd +++ b/doc/puppetfile.mkd @@ -305,3 +305,239 @@ mod 'apache', The given 'install\_path' can be an absolute path or a path relative to the base of the environment. Note that r10k will exit with an error if you attempt to set the 'path' option to a directory outside of the environment. + +## Puppetfile.\ + +When using r10k, Git branches correspond to Puppet environments. For small teams using trunk-based development, with one or maybe two deployment environments, it's reasonable to incorporate and deploy changes using standard `git merge` workflows. + +In large enterprise organizations with more complex deployment requirements, however, it may become untenable to try and manage change deployments across more than two environments using only `git merge`. For example, consider the following ordered tasks: + +1. Deploy change A to `devint` +2. Deploy change B to `devint` +3. Deploy change A to `staging` +4. Deploy change B to `staging` +5. Deploy change B to `production` +6. Deploy change C to `devint` +7. Deploy change C to `staging` +8. Deploy change C to `production` +9. Deploy change A to `production` + +This example uses only three deployment environments, and three changes. While it's not impossible to manage this kind of change decoulping in Git, it is prohibitively difficult. Furthermore, when this is managed entirely with Git merging, it can become difficult to look at any individual environment and understand which changes have been deployed to it. + +A trunk-based deployment workflow is considered a better practice and results in a simpler r10k workflow. Sometimes it's not possible to achieve that though, or getting to trunk-based deployment may take some time. + +R10k's Puppetfile.\ feature exists to help address complicated deployment use cases that are prohibitively difficult to manage with Git merging. + +### How it works + +Consider a use case with three deployment environments: + +1. devint +2. staging +3. production + +Choose a trunk branch. The Puppetfile.\ pattern extends the normal Git/r10k use case. In this example, we'll use the traditional git "master" branch as our trunk. + +Development of features occurs as per normal in the master branch. Its contents look something like this: + +``` +(master) +├── Puppetfile +├── environment.conf +├── hiera.yaml +└── site-modules + ├── profile + └── role +``` + +Perform normal development against this branch. It does not necessarily represent an important Puppet environment, though of course it is possible to perform canary agent runs against it using `puppet agent -t --environment=master`. + +The Puppetfile in this branch should list all of the modules deployed which _do not need_ decoupled change deployment. + +Suppose that Team C owns a Puppet module, team\_c, which needs changes deployed at an odd cadence across the three deployment environments. The Puppetfile should list modules such as ntp, stdlib, and so forth, but it **should not** list Team C's module. + +For example, the Puppetfile may look something like this: + +``` +# Puppetfile + +mod 'puppetlabs-ntp', '6.0.0' +``` + +In order to enable decoupled change deployment, Team C's module will be defined independently per-environment in the Puppetfile.\. + +Because Git branches correspond to Puppet environments, there is a Git branch for devint, staging, and production. However, these branches are special. No merging is ever performed on these branches, and they can only contain a single file. Here's what these branches look like: + +``` +(devint) +└── Puppetfile.DEVINT +``` + +``` +(staging) +└── Puppetfile.STAGING +``` + +``` +(production) +└── Puppetfile.PRODUCTION +``` + +The Puppetfile.\ is metadata that tells r10k how to deploy the environment. It configures two things: + +1. Which version of Puppet code to deploy from the trunk branch +2. Which decoupled modules to deploy at which versions + +The Puppetfile.DEVINT file may look something like this: + +``` +# Puppetfile.DEVINT + +environment 'devint', + :branch => 'master' + +mod 'team_c', + :git => 'git:///team_c.git', + :ref => 'v22' +``` + +The Puppetfile.PRODUCTION may look something like this: + +``` +# Puppetfile.PRODUCTION + +environment 'production', + :tag => 'release-302' + +mod 'team_c', + :git => 'git:///team_c.git', + :ref => 'v21' +``` + +When r10k deploys one of these environments, this is what the Puppet environment will look like on the master (using devint as the example): + +``` +/etc/puppetlabs/code/environments/devint +├── Puppetfile +├── Puppetfile.DEVINT +├── environment.conf +├── hiera.yaml +├── modules +│   ├── ntp +│   └── team_c +└── site-modules + ├── profile + └── role +``` + +Note that modules from the Puppetfile AND modules from the Puppetfile.\ are deployed. + +### Deployment + +To deploy changes to an environment defined using Puppetfile.\, edit the Puppetfile.\ file (in its corresponding Git branch). + +To deploy a module release, edit the appropriate `mod` directive with the version to deploy. + +To deploy an environment code release, edit the `environment` directive with the version to deploy. + +A manual deployment of the named environment may be performed at any time in the normal way (e.g. `r10k deploy environment devint`). If any references in the Puppetfile point to a branch this will result in the environment being updated to reflect any changes in referenced branch(es) since the last deploy. + +### Requirements + +To use the Puppetfile.\ feature, a Git branch must: + +* Contain one, and only one, committed file: Puppetfile.\ +* The \ component of the file name MUST be the upper-case name of the branch it is committed to +* The name given in the environment directive of Puppetfile.\ MUST match the name of the branch it is committed to + +### Environment directive + +The Puppetfile.\ files should contain a special "environment" directive. This tells r10k which version of code to deploy to the named environment. The environment directive behaves similarly to how the mod directive behaves when using a Git source. + +``` +# Puppetfile.PRODUCTION + +environment 'production', + :ref => '13.0.2' +``` + +The `environment` directive accepts two arguments. + +#### `:ref` + +As in a module, `:ref` may refer to a git tag, commit, or branch to deploy. + +#### `:git` + +By default, `environment` assumes you want to check out a different ref in the control repo. Using the `:git` parameter, it is possible to link to and deploy an environment code version from a separate repository entirely. For example: + +```ruby +# Puppetfile.PRODUCTION + +environment 'production', + :git => 'git://github.com/reidmv/puppet-control-repo.git', + :ref => 'b05b956' +``` + +``` +[root@master:~] # r10k deploy environment production -pv +INFO -> Using Puppetfile '/Users/reidmv/tmp/environments/production/Puppetfile' +INFO -> Deploying environment /etc/puppetlabs/code-staging/environments/production +INFO -> Environment production is now at b8fabbeac16374cfdeb5596c87edcedd144fafb9 +INFO -> Puppetfile at b8fabbeac16374cfdeb5596c87edcedd144fafb9 redirects to git://github.com/reidmv/puppet-control-repo.git "b05b956" +INFO -> Environment production is now at b05b956a972358a8f649f66f0993c0f0bdce9c5a +INFO -> Deploying Puppetfile content /etc/puppetlabs/code-staging/environments/production/site +INFO -> Deploying Puppetfile content /etc/puppetlabs/code-staging/environments/production/modules/stdlib +INFO -> Deploying Puppetfile content /etc/puppetlabs/code-staging/environments/production/modules/remote_file +INFO -> Deploying Puppetfile content /etc/puppetlabs/code-staging/environments/production/modules/concat +... +``` + +#### Example uses + +Users of this feature might implement a clean separation of development and stable deployment using separate r10k sources configured as follows. + +```yaml +--- +cachedir: '/var/cache/r10k' + +sources: + stable: + remote: 'git://github.com/reidmv/control-repo.git' + prefix: false + development: + remote: 'git://github.com/reidmv/puppet-environment.git' + prefix: 'git' +``` + +The "stable" source above is a typical control repo, where branches correspond to named environments. Every branch in this repo though is a "link" to either a branch or a code version from the puppet-environment.git repo. + +the "development" source, puppet-environment.git, has a typical git master branch, and feature branches for work in progress. Every branch in puppet-environment.git is available as a Puppet environment, but has the "git\_\*" prefix. For example, "git\_master". + +In the control-repo, an early environment such as development might be pinned to the master branch of puppet-environment.git. + +```ruby +#! Puppetfile +#============================================================================= +# ENVIRONMENT: development +#============================================================================= + +environment 'development', + :git => 'git://github.com/reidmv/puppet-environment.git', + :ref => 'master' +``` + +Higher tier environments are defined the same way, but likely pinned to commits or tags. E.g. + +```ruby +#! Puppetfile +#============================================================================= +# ENVIRONMENT: staging +#============================================================================= + +environment 'staging', + :git => 'git://github.com/reidmv/puppet-environment.git', + :ref => '2.1' +``` + +This pattern results in named environments without prefixes being defined and controlled in a deployment-centric workflow, while developers working on new features retain the same benefits they're used to; the main difference when compared to traditional development-only workflows is that Puppet environments that are deployed for development use are clearly identifyable by their "git\_\*" prefixes. diff --git a/lib/r10k/action/deploy/environment.rb b/lib/r10k/action/deploy/environment.rb index 9a221284d..b9690f05e 100644 --- a/lib/r10k/action/deploy/environment.rb +++ b/lib/r10k/action/deploy/environment.rb @@ -82,18 +82,28 @@ def visit_environment(environment) started_at = Time.new - status = environment.status logger.info _("Deploying environment %{env_path}") % {env_path: environment.path} - environment.sync - logger.info _("Environment %{env_dir} is now at %{env_signature}") % {env_dir: environment.dirname, env_signature: environment.signature} - - if status == :absent || @puppetfile - if status == :absent - logger.debug(_("Environment %{env_dir} is new, updating all modules") % {env_dir: environment.dirname}) + # Sync the environment and redirect once if necessary. Use &Proc.new + # to forward this method's block to the sync_environment! method. + if sync_environment!(environment, &Proc.new) == :redirect + + # Preserve the environment Puppetfile, sync the redirected + # environment, and restore the environment Puppetfile for + # reference. + envpf_path = environment.puppetfile.puppetfile_path + envpf_name = environment.puppetfile.puppetfile_name + envpf_contents = environment.puppetfile.puppetfile_contents + + # Sync to the redirected environment, but don't allow a repeat + # redirection + if sync_environment!(environment, &Proc.new) != :done + raise R10K::Error.new('Redirect can only be performed once per environment, per deploy') end - yield + # Restore the environment Puppetfile to disk so that all of the + # artifacts that contributed to building an environment are visible + File.open(envpf_path, 'w') { |file| file.write(envpf_contents) } end if @purge_levels.include?(:environment) @@ -108,9 +118,39 @@ def visit_environment(environment) write_environment_info!(environment, started_at, @visit_ok) end + def sync_environment!(environment) + status = environment.status + environment.sync + + logger.info _("Environment %{env_dir} is now at %{env_signature}") % {env_dir: environment.dirname, env_signature: environment.signature} + logger.debug(_("Environment %{env_dir} is new, updating all modules") % {env_dir: environment.dirname}) if status == :absent + + if (@puppetfile || status == :absent) && (yield == :redirect) + redirect = environment.puppetfile.environment_redirect + environment.ref = redirect[:ref] + environment.remote = redirect[:git] if redirect[:git] + :redirect + else + :done + end + end + def visit_puppetfile(puppetfile) puppetfile.load + if puppetfile.environment_redirect? + signature = puppetfile.environment.signature + newremote = puppetfile.environment_redirect[:git] + newref = puppetfile.environment_redirect[:ref] + remotestr = newremote ? "#{newremote} " : '' + logger.info(_('Puppetfile at %{signature} redirects to %{remote}"%{ref}"') % {signature: signature, remote: remotestr, ref: newref}) + unless puppetfile.modules.empty? + names = puppetfile.modules.map{ |m| m.name }.join(',') + logger.info(_('Loaded modules from Puppetfile at %{signature}: %{names}') % {signature: signature, names: names}) + end + return :redirect + end + yield if @purge_levels.include?(:puppetfile) diff --git a/lib/r10k/environment/base.rb b/lib/r10k/environment/base.rb index 9741dc58a..56fc5c2d5 100644 --- a/lib/r10k/environment/base.rb +++ b/lib/r10k/environment/base.rb @@ -115,6 +115,7 @@ def purge_exclusions list = [File.join(@full_path, '.r10k-deploy.json')].to_set list += @puppetfile.managed_directories + list += @puppetfile.puppetfile_paths list += @puppetfile.desired_contents.flat_map do |item| desired_tree = [] diff --git a/lib/r10k/environment/git.rb b/lib/r10k/environment/git.rb index 4830f1912..a7eb9f0c2 100644 --- a/lib/r10k/environment/git.rb +++ b/lib/r10k/environment/git.rb @@ -16,7 +16,7 @@ class R10K::Environment::Git < R10K::Environment::Base # @!attribute [r] ref # @return [String] The git reference to use for this environment - attr_reader :ref + attr_accessor :ref # @!attribute [r] repo # @api private @@ -40,6 +40,11 @@ def initialize(name, basedir, dirname, options = {}) @repo = R10K::Git::StatefulRepository.new(@remote, @basedir, @dirname) end + def remote=(addr) + @remote = addr + @repo = R10K::Git::StatefulRepository.new(@remote, @basedir, @dirname) + end + # Clone or update the given Git environment. # # If the environment is being created for the first time, it will diff --git a/lib/r10k/git/rugged/thin_repository.rb b/lib/r10k/git/rugged/thin_repository.rb index 80392aaac..ec328d28d 100644 --- a/lib/r10k/git/rugged/thin_repository.rb +++ b/lib/r10k/git/rugged/thin_repository.rb @@ -3,10 +3,10 @@ require 'r10k/git/rugged/cache' class R10K::Git::Rugged::ThinRepository < R10K::Git::Rugged::WorkingRepository - def initialize(basedir, dirname, cache_repo) + def initialize(basedir, dirname, gitdirname, cache_repo) @cache_repo = cache_repo - super(basedir, dirname) + super(basedir, dirname, gitdirname) end # Clone this git repository @@ -29,8 +29,9 @@ def clone(remote, opts = {}) # update 'objects/info/alternates' with the path. We don't actually # fetch any objects because we don't need them, and we don't actually # use any refs in this repository so we skip all those steps. - ::Rugged::Repository.init_at(@path.to_s, false) - @_rugged_repo = ::Rugged::Repository.new(@path.to_s, :alternates => [cache_objects_dir]) + ::Rugged::Repository.init_at(@git_dir.to_s, true) + @_rugged_repo = ::Rugged::Repository.new(@git_dir.to_s, :alternates => [cache_objects_dir]) + @_rugged_repo.workdir = @path.to_s alternates << cache_objects_dir with_repo do |repo| diff --git a/lib/r10k/git/rugged/working_repository.rb b/lib/r10k/git/rugged/working_repository.rb index f78391ff1..eeeb4a930 100644 --- a/lib/r10k/git/rugged/working_repository.rb +++ b/lib/r10k/git/rugged/working_repository.rb @@ -4,15 +4,16 @@ class R10K::Git::Rugged::WorkingRepository < R10K::Git::Rugged::BaseRepository - # @return [Pathname] The path to the Git repository inside of this directory - def git_dir - @path + '.git' - end + # @attribute [r] git_dir + # @return [Pathname] + attr_reader :git_dir # @param basedir [String] The base directory of the Git repository # @param dirname [String] The directory name of the Git repository - def initialize(basedir, dirname) - @path = Pathname.new(File.join(basedir, dirname)) + # @param gitdirname [String] The relative name of the gitdir + def initialize(basedir, dirname, gitdirname = '.git') + @path = Pathname.new(File.join(basedir, dirname)).cleanpath + @git_dir = Pathname.new(File.join(basedir, dirname, gitdirname)).cleanpath end # Clone this git repository @@ -34,13 +35,15 @@ def clone(remote, opts = {}) # repository. However alternate databases can be handled when an existing # repository is loaded, so loading a cloned repo will correctly use # alternate object database. - options = {:credentials => credentials} + options = {:credentials => credentials, :bare => true} options.merge!(:alternates => [File.join(opts[:reference], 'objects')]) if opts[:reference] proxy = R10K::Git.get_proxy_for_remote(remote) R10K::Git.with_proxy(proxy) do - @_rugged_repo = ::Rugged::Repository.clone_at(remote, @path.to_s, options) + @_rugged_repo = ::Rugged::Repository.clone_at(remote, @git_dir.to_s, options) + @_rugged_repo.workdir = @path.to_s + @_rugged_repo.checkout(@_rugged_repo.head.canonical_name) end if opts[:reference] @@ -53,7 +56,7 @@ def clone(remote, opts = {}) checkout(opts[:ref]) end rescue Rugged::SshError, Rugged::NetworkError => e - raise R10K::Git::GitError.new(e.message, :git_dir => git_dir, :backtrace => e.backtrace) + raise R10K::Git::GitError.new(e.message, :git_dir => @git_dir, :backtrace => e.backtrace) end # Check out the given Git ref @@ -66,7 +69,7 @@ def checkout(ref, opts = {}) if sha logger.debug2 { _("Checking out ref '%{ref}' (resolved to SHA '%{sha}') in repository %{path}") % {ref: ref, sha: sha, path: @path} } else - raise R10K::Git::GitError.new("Unable to check out unresolvable ref '#{ref}'", git_dir: git_dir) + raise R10K::Git::GitError.new("Unable to check out unresolvable ref '#{ref}'", git_dir: @git_dir) end # :force defaults to true @@ -98,7 +101,7 @@ def fetch(remote_name = 'origin') report_transfer(results, remote) rescue Rugged::SshError, Rugged::NetworkError => e - raise R10K::Git::GitError.new(e.message, :git_dir => git_dir, :backtrace => e.backtrace) + raise R10K::Git::GitError.new(e.message, :git_dir => @git_dir, :backtrace => e.backtrace) end def exist? @@ -110,7 +113,7 @@ def head end def alternates - R10K::Git::Alternates.new(git_dir) + R10K::Git::Alternates.new(@git_dir) end def origin @@ -133,13 +136,15 @@ def dirty? private def with_repo - if @_rugged_repo.nil? && git_dir.exist? + if @_rugged_repo.nil? && @git_dir.exist? setup_rugged_repo end super end def setup_rugged_repo - @_rugged_repo = ::Rugged::Repository.new(@path.to_s, :alternates => alternates.to_a) + @_rugged_repo = ::Rugged::Repository.new(@git_dir.to_s, :alternates => alternates.to_a) + @_rugged_repo.workdir = @path.to_s + @_rugged_repo end end diff --git a/lib/r10k/git/shellgit/thin_repository.rb b/lib/r10k/git/shellgit/thin_repository.rb index bccb134ce..884fc6573 100644 --- a/lib/r10k/git/shellgit/thin_repository.rb +++ b/lib/r10k/git/shellgit/thin_repository.rb @@ -8,9 +8,9 @@ # making new clones very lightweight. class R10K::Git::ShellGit::ThinRepository < R10K::Git::ShellGit::WorkingRepository - def initialize(basedir, dirname, cache_repo) + def initialize(basedir, dirname, gitdirname, cache_repo) @cache_repo = cache_repo - super(basedir, dirname) + super(basedir, dirname, gitdirname) end # Clone this git repository @@ -36,11 +36,11 @@ def fetch(remote = 'cache') # @return [String] The origin remote URL def cache - git(['config', '--get', 'remote.cache.url'], :path => @path.to_s, :raise_on_fail => false).stdout + git(['config', '--get', 'remote.cache.url'], :raise_on_fail => false).stdout end def tracked_paths(ref="HEAD") - git(['ls-tree', '-t', '-r', '--name-only', ref], :path => @path.to_s).stdout.split("\n") + git(['ls-tree', '-t', '-r', '--name-only', ref]).stdout.split("\n") end private diff --git a/lib/r10k/git/shellgit/working_repository.rb b/lib/r10k/git/shellgit/working_repository.rb index e7926d305..a887528f2 100644 --- a/lib/r10k/git/shellgit/working_repository.rb +++ b/lib/r10k/git/shellgit/working_repository.rb @@ -9,13 +9,20 @@ class R10K::Git::ShellGit::WorkingRepository < R10K::Git::ShellGit::BaseReposito # @return [Pathname] attr_reader :path - # @return [Pathname] The path to the Git directory inside of this repository - def git_dir - @path + '.git' + # @attribute [r] git_dir + # @return [Pathname] + attr_reader :git_dir + + def initialize(basedir, dirname, gitdirname = '.git') + @path = Pathname.new(File.join(basedir, dirname)).cleanpath + @git_dir = Pathname.new(File.join(basedir, dirname, gitdirname)).cleanpath end - def initialize(basedir, dirname) - @path = Pathname.new(File.join(basedir, dirname)) + def git(cmd, opts = {}) + work_tree = opts.delete(:path) || @path.to_s + path_opts = {:git_dir => @git_dir.to_s, :work_tree => work_tree} + + super(cmd, path_opts.merge(opts)) end # Clone this git repository @@ -28,19 +35,22 @@ def initialize(basedir, dirname) # # @return [void] def clone(remote, opts = {}) - argv = ['clone', remote, @path.to_s] + clone_argv = ['clone', '--bare', '--single-branch', '-c', 'remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*', remote, @git_dir.to_s] if opts[:reference] - argv += ['--reference', opts[:reference]] + clone_argv += ['--reference', opts[:reference]] end proxy = R10K::Git.get_proxy_for_remote(remote) R10K::Git.with_proxy(proxy) do - git argv + git clone_argv + git ['fetch', 'origin', '--prune'] end if opts[:ref] checkout(opts[:ref]) + else + checkout('HEAD') end end @@ -56,7 +66,7 @@ def checkout(ref, opts = {}) argv << '--force' end - git argv, :path => @path.to_s + git argv end def fetch(remote_name='origin') @@ -64,7 +74,7 @@ def fetch(remote_name='origin') proxy = R10K::Git.get_proxy_for_remote(remote) R10K::Git.with_proxy(proxy) do - git ['fetch', remote_name, '--prune'], :path => @path.to_s + git ['fetch', remote_name, '--prune'] end end @@ -83,7 +93,7 @@ def alternates # @return [String] The origin remote URL def origin - result = git(['config', '--get', 'remote.origin.url'], :path => @path.to_s, :raise_on_fail => false) + result = git(['config', '--get', 'remote.origin.url'], :raise_on_fail => false) if result.success? result.stdout end @@ -91,7 +101,7 @@ def origin # does the working tree have local modifications to tracked files? def dirty? - result = git(['diff-index', '--exit-code', '--name-only', 'HEAD'], :path => @path.to_s, :raise_on_fail => false) + result = git(['diff-index', '--exit-code', '--name-only', 'HEAD'], :raise_on_fail => false) if result.exit_code != 0 dirty_files = result.stdout.split('\n') @@ -100,7 +110,7 @@ def dirty? logger.debug(_("Found local modifications in %{file_path}" % {file_path: File.join(@path, file)})) # Do this in a block so that the extra subprocess only gets invoked when needed. - logger.debug1 { git(['diff-index', '-p', 'HEAD', file], :path => @path.to_s, :raise_on_fail => false).stdout } + logger.debug1 { git(['diff-index', '-p', 'HEAD', file], :raise_on_fail => false).stdout } end return true diff --git a/lib/r10k/git/stateful_repository.rb b/lib/r10k/git/stateful_repository.rb index 981ac0ff6..69531e982 100644 --- a/lib/r10k/git/stateful_repository.rb +++ b/lib/r10k/git/stateful_repository.rb @@ -20,10 +20,10 @@ class R10K::Git::StatefulRepository # @param remote [String] The git remote to use for the repo # @param basedir [String] The path containing the Git repo # @param dirname [String] The directory name of the Git repo - def initialize(remote, basedir, dirname) + def initialize(remote, basedir, dirname, gitdirname = '.git') @remote = remote @cache = R10K::Git.cache.generate(@remote) - @repo = R10K::Git.thin_repository.new(basedir, dirname, @cache) + @repo = R10K::Git.thin_repository.new(basedir, dirname, gitdirname, @cache) end def resolve(ref) @@ -43,12 +43,12 @@ def sync(ref, force=true) workdir_status = status(ref) case workdir_status - when :absent + when :absent, :uninitialized logger.debug(_("Cloning %{repo_path} and checking out %{ref}") % {repo_path: @repo.path, ref: ref }) @repo.clone(@remote, {:ref => sha}) when :mismatched logger.debug(_("Replacing %{repo_path} and checking out %{ref}") % {repo_path: @repo.path, ref: ref }) - @repo.path.rmtree + @repo.git_dir.rmtree @repo.clone(@remote, {:ref => sha}) when :outdated logger.debug(_("Updating %{repo_path} to %{ref}") % {repo_path: @repo.path, ref: ref }) @@ -70,7 +70,7 @@ def status(ref) if !@repo.exist? :absent elsif !@repo.git_dir.exist? - :mismatched + :uninitialized elsif !@repo.git_dir.directory? :mismatched elsif !(@repo.origin == @remote) diff --git a/lib/r10k/puppetfile.rb b/lib/r10k/puppetfile.rb index 38d3ea77a..ed7db7663 100644 --- a/lib/r10k/puppetfile.rb +++ b/lib/r10k/puppetfile.rb @@ -25,10 +25,18 @@ class Puppetfile # @return [String] The directory to install the modules #{basedir}/modules attr_reader :moduledir + # @!attrbute [r] puppetfile_name + # @return [String] The name of the Puppetfile + attr_reader :puppetfile_name + # @!attrbute [r] puppetfile_path # @return [String] The path to the Puppetfile attr_reader :puppetfile_path + # @!attrbute [r] puppetfile_paths + # @return [String] An array of Puppetfile paths, including each Puppetfile consulted + attr_reader :puppetfile_paths + # @!attribute [rw] environment # @return [R10K::Environment] Optional R10K::Environment that this Puppetfile belongs to. attr_accessor :environment @@ -46,20 +54,43 @@ def initialize(basedir, moduledir = nil, puppetfile_path = nil, puppetfile_name @basedir = basedir @force = force || false @moduledir = moduledir || File.join(basedir, 'modules') - @puppetfile_name = puppetfile_name || 'Puppetfile' - @puppetfile_path = puppetfile_path || File.join(basedir, @puppetfile_name) + @input_pf_name = puppetfile_name || 'Puppetfile' - logger.info _("Using Puppetfile '%{puppetfile}'") % {puppetfile: @puppetfile_path} + @puppetfile_name = @input_pf_name + @puppetfile_path = puppetfile_path || File.join(basedir, @input_pf_name) + @puppetfile_paths = [] @modules = [] + @environment_redirect = nil @managed_content = {} @forge = 'forgeapi.puppetlabs.com' + logger.info _("Using Puppetfile '%{puppetfile}'") % {puppetfile: @puppetfile_path} + @loaded = false end def load + # Calculate the environment-specific Puppetfile paths; e.g. + # "Puppetfile.PRODUCTION" + env_pf_name = @environment ? [@input_pf_name, @environment.name.upcase].join('.') : nil + env_pf_path = @environment ? File.join(basedir, env_pf_name) : nil + + # If environment-specific Puppetfile paths apply, consult the + # environment-specific Puppetfile. Otherwise consult the standard + # Puppetfile. + name_and_path = if ( env_pf_path && + environment.repo.tracked_paths == [env_pf_name] ) + [env_pf_name, env_pf_path] + else + [@input_pf_name, File.join(basedir, @input_pf_name)] + end + + @puppetfile_name, @puppetfile_path = name_and_path + + # Now that the appropriate paths have been set, try to load the file if File.readable? @puppetfile_path + @puppetfile_paths << @puppetfile_path unless @puppetfile_paths.include?(@pupetfile_path) self.load! else logger.debug _("Puppetfile %{path} missing or unreadable") % {path: @puppetfile_path.inspect} @@ -67,14 +98,35 @@ def load end def load! + @environment_redirect = nil + logger.debug _("Reading Puppetfile from %{path}") % {path: @puppetfile_path.inspect} dsl = R10K::Puppetfile::DSL.new(self) dsl.instance_eval(puppetfile_contents, @puppetfile_path) + validate_environment_redirect(@environment_redirect, @modules) validate_no_duplicate_names(@modules) @loaded = true rescue SyntaxError, LoadError, ArgumentError, NameError => e raise R10K::Error.wrap(e, _("Failed to evaluate %{path}") % {path: @puppetfile_path}) end + # @param [Hash] environment + # @param [Array] modules + def validate_environment_redirect(redirect, modules) + return if redirect.nil? + + if environment.nil? + raise R10K::Error.new('Puppetfile `environment` method may only be used to deploy environments! Cannot be used with `puppetfile install`') + end + + unless environment.name == redirect[:name] + raise R10K::Error.new('The name given to `environment` must match the source branch name') + end + + unless environment.repo.is_a?(R10K::Git::StatefulRepository) + raise R10K::Error.new('`environment` can only be used with a Git source') + end + end + # @param [Array] modules def validate_no_duplicate_names(modules) dupes = modules @@ -89,6 +141,19 @@ def validate_no_duplicate_names(modules) end end + # @return [Boolean] Whether or not the Puppetfile redirects to a new environment + def environment_redirect? + !@environment_redirect.nil? + end + + def environment_redirect + if @environment_redirect.nil? + nil + else + @environment_redirect.reject { |key| key == :name } + end + end + # @param [String] forge def set_forge(forge) @forge = forge @@ -122,6 +187,16 @@ def add_module(name, args) @modules << mod end + def set_environment_redirect(name, args) + unless [args[:ref], args[:tag], args[:branch]].compact.size == 1 + raise R10K::Error.new('Must specify a ref when using `environment`!') + end + + ref = args[:ref] || args[:tag] || args[:branch] + + @environment_redirect = {:name => name, :ref => ref, :git => args[:git]} + end + include R10K::Util::Purgeable def managed_directories @@ -159,12 +234,12 @@ def accept(visitor) end end - private - def puppetfile_contents File.read(@puppetfile_path) end + private + def resolve_install_path(path) pn = Pathname.new(path) @@ -197,6 +272,10 @@ def initialize(librarian) @librarian = librarian end + def environment(name, args = nil) + @librarian.set_environment_redirect(name, args) + end + def mod(name, args = nil) @librarian.add_module(name, args) end diff --git a/spec/fixtures/unit/puppetfile/environment-mod-conflict/Puppetfile b/spec/fixtures/unit/puppetfile/environment-mod-conflict/Puppetfile new file mode 100644 index 000000000..7fc0d8cc7 --- /dev/null +++ b/spec/fixtures/unit/puppetfile/environment-mod-conflict/Puppetfile @@ -0,0 +1,2 @@ +environment 'myenv', ref: '1.2.3' +mod 'lwf-remote_file', '1.1.3' diff --git a/spec/integration/git/rugged/thin_repository_spec.rb b/spec/integration/git/rugged/thin_repository_spec.rb index 145c2d00e..794509eaa 100644 --- a/spec/integration/git/rugged/thin_repository_spec.rb +++ b/spec/integration/git/rugged/thin_repository_spec.rb @@ -6,9 +6,11 @@ let(:dirname) { 'working-repo' } + let(:gitdirname) { '.git' } + let(:cacherepo) { R10K::Git::Rugged::Cache.generate(remote) } - subject { described_class.new(basedir, dirname, cacherepo) } + subject { described_class.new(basedir, dirname, gitdirname, cacherepo) } it_behaves_like "a git thin repository" end diff --git a/spec/integration/git/shellgit/thin_repository_spec.rb b/spec/integration/git/shellgit/thin_repository_spec.rb index a27336ece..fe3c032f2 100644 --- a/spec/integration/git/shellgit/thin_repository_spec.rb +++ b/spec/integration/git/shellgit/thin_repository_spec.rb @@ -6,9 +6,11 @@ let(:dirname) { 'working-repo' } + let(:gitdirname) { '.git' } + let(:cacherepo) { R10K::Git::ShellGit::Cache.generate(remote) } - subject { described_class.new(basedir, dirname, cacherepo) } + subject { described_class.new(basedir, dirname, gitdirname, cacherepo) } it_behaves_like "a git thin repository" end diff --git a/spec/integration/git/stateful_repository_spec.rb b/spec/integration/git/stateful_repository_spec.rb index b183e4e13..46efc0118 100644 --- a/spec/integration/git/stateful_repository_spec.rb +++ b/spec/integration/git/stateful_repository_spec.rb @@ -6,9 +6,10 @@ include_context 'Git integration' let(:dirname) { 'working-repo' } + let(:gitdirname) { '.git' } let(:cacherepo) { R10K::Git.cache.generate(remote) } - let(:thinrepo) { R10K::Git.thin_repository.new(basedir, dirname, cacherepo) } + let(:thinrepo) { R10K::Git.thin_repository.new(basedir, dirname, gitdirname, cacherepo) } let(:ref) { '0.9.x' } subject { described_class.new(remote, basedir, dirname) } @@ -21,16 +22,16 @@ end describe "when the directory is not a git repository" do - it "is mismatched" do + it "is uninitialized" do thinrepo.path.mkdir - expect(subject.status(ref)).to eq :mismatched + expect(subject.status(ref)).to eq :uninitialized end end describe "when the directory has a .git file" do it "is mismatched" do thinrepo.path.mkdir - File.open("#{thinrepo.path}/.git", "w") {} + File.open("#{thinrepo.path}/#{gitdirname}", "w") {} expect(subject.status(ref)).to eq :mismatched end end diff --git a/spec/shared-examples/git/working_repository.rb b/spec/shared-examples/git/working_repository.rb index 0421657f4..42f815294 100644 --- a/spec/shared-examples/git/working_repository.rb +++ b/spec/shared-examples/git/working_repository.rb @@ -174,7 +174,7 @@ context "with force = false" do it "should not revert changes in managed files" do - subject.checkout(subject.head, {:force => false}) + expect { subject.checkout(subject.head, {:force => false}).to raise_error(Rugged::CheckoutError) } expect(File.read(File.join(subject.path, 'README.markdown')).include?('local modifications!')).to eq true end end diff --git a/spec/unit/puppetfile_spec.rb b/spec/unit/puppetfile_spec.rb index 845abd2df..c94b07247 100644 --- a/spec/unit/puppetfile_spec.rb +++ b/spec/unit/puppetfile_spec.rb @@ -129,6 +129,18 @@ end end + describe "using `environment` in the Puppetfile" do + it "should accept environment with a ref" do + expect { subject.set_environment_redirect('myenv', {ref: '1.2.3'}) }.to change { subject.environment_redirect } + expect(subject.environment_redirect[:ref]).to include('1.2.3') + end + + it "should accept environment with a ref and a remote" do + expect { subject.set_environment_redirect('myenv', {ref: '1.2.3', git: 'git:///new'}) }.to change { subject.environment_redirect } + expect(subject.environment_redirect[:git]).to include('git:///new') + end + end + describe "#purge_exclusions" do let(:managed_dirs) { ['dir1', 'dir2'] }