Skip to content

Commit

Permalink
Add option to generate setfile with multiple hosts and roles
Browse files Browse the repository at this point in the history
  • Loading branch information
@h-haaks
h-haaks committed Apr 17, 2024
1 parent ca6a320 commit 670fbea
Show file tree
Hide file tree
Showing 6 changed files with 298 additions and 125 deletions.
149 changes: 112 additions & 37 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,23 +16,23 @@ To get outputs [usable in Github Actions](https://docs.github.com/en/free-pro-te

```console
$ metadata2gha
puppet_major_versions=[{"name":"Puppet 7","value":7,"collection":"puppet7"},{"name":"Puppet 6","value":6,"collection":"puppet6"}]
puppet_unit_test_matrix=[{"puppet":7,"ruby":"2.7"},{"puppet":6,"ruby":"2.5"}]
github_action_test_matrix=[{"setfile":{"name":"Debian 11","value":"debian11-64"},"puppet":{"name":"Puppet 7","value":7,"collection":"puppet7"}},{"setfile":{"name":"Debian 11","value":"debian11-64"},"puppet":{"name":"Puppet 6","value":6,"collection":"puppet6"}}]
puppet_major_versions=[{"name":"Puppet 8","value":8,"collection":"puppet8"},{"name":"Puppet 7","value":7,"collection":"puppet7"}]
puppet_unit_test_matrix=[{"puppet":8,"ruby":"3.2"},{"puppet":7,"ruby":"2.7"}]
puppet_beaker_test_matrix=[{"name":"Puppet 8 - Debian 12","env":{"BEAKER_PUPPET_COLLECTION":"puppet8","BEAKER_SETFILE":"debian12-64{hostname=debian12-64-puppet8}"}},{"name":"Puppet 7 - Debian 12","env":{"BEAKER_PUPPET_COLLECTION":"puppet7","BEAKER_SETFILE":"debian12-64{hostname=debian12-64-puppet7}"}}]
```

Puppet major versions formatted for readability:
```json
[
{
"name": "Puppet 8",
"value": 8,
"collection": "puppet8"
},
{
"name": "Puppet 7",
"value": 7,
"collection": "puppet7"
},
{
"name": "Puppet 6",
"value": 6,
"collection": "puppet6"
}
]
```
Expand All @@ -41,45 +41,37 @@ Puppet unit test matrix formatted for readability:
```json
[
{
"puppet": 7,
"ruby": "2.7"
"puppet": 8,
"ruby": "3.2"
},
{
"puppet": 6,
"ruby": "2.5"
"puppet": 7,
"ruby": "2.7"
}
]
```

GitHub Action test matrix formatted for readability
Beaker test matrix formatted for readability
```json
[
{
"setfile": {
"name": "Debian 11",
"value": "debian11-64"
},
"puppet": {
"name": "Puppet 7",
"value": 7,
"collection": "puppet7"
"name": "Puppet 8 - Debian 12",
"env": {
"BEAKER_PUPPET_COLLECTION": "puppet8",
"BEAKER_SETFILE": "debian12-64{hostname=debian12-64-puppet8}"
}
},
{
"setfile": {
"name": "Debian 11",
"value": "debian11-64"
},
"puppet": {
"name": "Puppet 6",
"value": 6,
"collection": "puppet6"
"name": "Puppet 7 - Debian 12",
"env": {
"BEAKER_PUPPET_COLLECTION": "puppet7",
"BEAKER_SETFILE": "debian12-64{hostname=debian12-64-puppet7}"
}
}
]
```

It is also possible to specify the path to metadata.json and customize the setfiles. For example, to ensure the setfiles use FQDNs and apply the [systemd PIDFile workaround under docker](https://github.com/docker/for-linux/issues/835). This either means either using an older image (CentOS 7, Ubuntu 16.04) or skipping (CentOS 8).
It is possible to specify the path to metadata.json and customize the setfiles. For example, to ensure the setfiles use FQDNs and apply the [systemd PIDFile workaround under docker](https://github.com/docker/for-linux/issues/835). This either means either using an older image (CentOS 7, Ubuntu 16.04) or skipping (CentOS 8).

```console
$ metadata2gha --use-fqdn --pidfile-workaround true /path/to/metadata.json
Expand All @@ -89,21 +81,104 @@ This results in the following JSON data
```json
[
{
"name": "CentOS 7",
"value": "centos7-64{hostname=centos7-64.example.com,image=centos:7.6.1810}"
"name": "Puppet 7 - CentOS 7",
"env": {
"BEAKER_PUPPET_COLLECTION": "puppet7",
"BEAKER_SETFILE": "centos7-64{hostname=centos7-64-puppet7.example.com,image=centos:7.6.1810}"
}
},
{
"name": "Debian 10",
"value": "debian10-64{hostname=debian10-64.example.com}"
"name": "Puppet 7 - Debian 12",
"env": {
"BEAKER_PUPPET_COLLECTION": "puppet7",
"BEAKER_SETFILE": "debian12-64{hostname=debian12-64-puppet7.example.com}"
}
},
{
"name": "Ubuntu 18.04",
"value": "ubuntu1804-64{hostname=ubuntu1804-64.example.com}"
"name": "Puppet 7 - Ubuntu 22.04",
"env": {
"BEAKER_PUPPET_COLLECTION": "puppet7",
"BEAKER_SETFILE": "ubuntu2204-64{hostname=ubuntu2204-64-puppet7.example.com}"
}
}
]
```

If you need multiple hosts in your integration tests this could be achived by using the --beaker_nodes_and_roles option

Option argument is '#HOST:ROLE,ROLE,...;#HOST:..;..' where
- hosts are separated by ';'
- host number and roles are separated by ':'
- Roles are separated by ','
If you don't need any extra roles use '1;2;..'

The 'master' and 'agent' roles will be automaticly added to the first host.
The 'agent' role will be automaticly added to the rest.

```console
$ metadata2gha --beaker_nodes_and_roles '1:role1,other1;2:role2,other2'
```

This results in the following JSON data
```json
[
{
"name": "Puppet 7 - Debian 12",
"env": {
"BEAKER_PUPPET_COLLECTION": "puppet7",
"BEAKER_SETFILE": "debian12-64role1,other1.ma{hostname=debian12-64-puppet7-1}-debian12-64role2,other2.a{hostname=debian12-64-puppet7-2}"
}
}
]
```

It is also possible to specify a comma separated list of operating systems as used in `metadata.json` (`CentOS,Ubuntu`).
If you need to Expand the matrix ie by product versions it could be achived by using the --beaker-facter option

Option argument is 'FACT:LABEL:VALUE,VALUE,..' where
- Fact, label and values are separated by ':'
- Values are separated by ','

```console
$ metadata2gha --beaker-facter 'mongodb_repo_version:MongoDB:4.4,5.0,6.0,7.0'
```

This results in the following JSON data
```json
[
{
"name": "Puppet 7 - Debian 12 - MongoDB 4.4",
"env": {
"BEAKER_PUPPET_COLLECTION": "puppet7",
"BEAKER_SETFILE": "debian12-64{hostname=debian12-64-puppet7}",
"BEAKER_FACTER_mongodb_repo_version": "4.4"
}
},
{
"name": "Puppet 7 - Debian 12 - MongoDB 5.0",
"env": {
"BEAKER_PUPPET_COLLECTION": "puppet7",
"BEAKER_SETFILE": "debian12-64{hostname=debian12-64-puppet7}",
"BEAKER_FACTER_mongodb_repo_version": "5.0"
}
},
{
"name": "Puppet 7 - Debian 12 - MongoDB 6.0",
"env": {
"BEAKER_PUPPET_COLLECTION": "puppet7",
"BEAKER_SETFILE": "debian12-64{hostname=debian12-64-puppet7}",
"BEAKER_FACTER_mongodb_repo_version": "6.0"
}
},
{
"name": "Puppet 7 - Debian 12 - MongoDB 7.0",
"env": {
"BEAKER_PUPPET_COLLECTION": "puppet7",
"BEAKER_SETFILE": "debian12-64{hostname=debian12-64-puppet7}",
"BEAKER_FACTER_mongodb_repo_version": "7.0"
}
}
]
```

## Work with the API

Expand Down
13 changes: 13 additions & 0 deletions bin/metadata2gha
View file
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,19 @@ OptionParser.new do |opts|
options[:beaker_facter] = [fact, label, values.split(',')]
end
end
opts.on('--beaker_nodes_and_roles #NODE:ROLES;#NODE:ROLES;...', 'Expand the setfile string to create multiple nodes with custom roles. Separate roles using commas') do |opt|
options[:beaker_nodes_and_roles] = {}
if opt != 'false'
opt.split(';').each do |node|
node_num, roles = node.split(':', 2)
options[:beaker_nodes_and_roles][node_num] = if roles
roles.split(',')
else
[]
end
end
end
end
end.parse!

filename = ARGV[0]
Expand Down
29 changes: 24 additions & 5 deletions lib/puppet_metadata/beaker.rb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -58,17 +58,18 @@ def adjusted_os(os)
#
# @return [nil] If no setfile is available
# @return [Array<(String, String)>] The beaker setfile description with a readable name
def os_release_to_setfile(os, release, use_fqdn: false, pidfile_workaround: false, domain: nil, puppet_version: nil)
def os_release_to_setfile(os, release, use_fqdn: false, pidfile_workaround: false, domain: nil, puppet_version: nil, nodes_and_roles: nil)
return unless os_supported?(os)

aos = adjusted_os(os)

name = "#{aos}#{release.tr('.', '')}-64"
hostname = (puppet_version.nil? && puppet_version != 'none') ? name : "#{name}-#{puppet_version}"
domain ||= 'example.com' if use_fqdn

hostname = (puppet_version.nil? || puppet_version == 'none') ? name : "#{name}-#{puppet_version}"
options = {}
options[:hostname] = "#{hostname}.#{domain}" if domain
if domain || (puppet_version && puppet_version != 'none')
options[:hostname] = domain ? "#{hostname}.#{domain}" : hostname
end

# Docker messes up cgroups and some systemd versions can't deal with
# that when PIDFile is used.
Expand All @@ -82,7 +83,25 @@ def os_release_to_setfile(os, release, use_fqdn: false, pidfile_workaround: fals

human_name = "#{os} #{release}"

[build_setfile(name, options), human_name]
if nodes_and_roles
names = []
nodes_and_roles.each do |node, roles|
roles.map!(&:strip)
n = "#{name}#{roles.join(',')}"
n += if names.empty?
# add master role to first node
'.ma'
else
'.a'
end
options[:hostname] = hostname + "-#{node.strip}" if nodes_and_roles.size > 1
options[:hostname] = "#{options[:hostname]}.#{domain}" if options[:hostname] && domain
names << build_setfile(n, options)
end
[names.join('-'), human_name]
else
[build_setfile(name, options), human_name]
end
end

# Return whether a Beaker setfile can be generated for the given OS
Expand Down
1 change: 1 addition & 0 deletions lib/puppet_metadata/github_actions.rb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,7 @@ def os_release_to_beaker_setfile(os, release, puppet_collection)
pidfile_workaround: options[:beaker_pidfile_workaround],
domain: options[:domain],
puppet_version: puppet_collection,
nodes_and_roles: options[:beaker_nodes_and_roles],
)
end
end
Expand Down
35 changes: 35 additions & 0 deletions spec/beaker_spec.rb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -57,5 +57,40 @@
it { expect(described_class.os_release_to_setfile('CentOS', '8', pidfile_workaround: ['CentOS'])).to be_nil }
end
end

describe 'domain' do
it {
expect(described_class.os_release_to_setfile('CentOS', '7', domain: 'mydomain.org',
use_fqdn: true)).to eq(['centos7-64{hostname=centos7-64.mydomain.org}', 'CentOS 7'])
}
end

describe 'puppet_version' do
[
['CentOS', '7', 'none', ['centos7-64', 'CentOS 7']],
['CentOS', '7', 'puppet7', ['centos7-64{hostname=centos7-64-puppet7}', 'CentOS 7']],
].each do |os, release, puppet_version, expected|
it { expect(described_class.os_release_to_setfile(os, release, puppet_version: puppet_version)).to eq(expected) }
end
end

describe 'nodes_and_roles' do
[
['CentOS', '7', { '1' => ['role1', 'role2'] }, ['centos7-64role1,role2.ma', 'CentOS 7']],
['CentOS', '7', { '1' => [], '2' => [] }, ['centos7-64.ma{hostname=centos7-64-1}-centos7-64.a{hostname=centos7-64-2}', 'CentOS 7']],
['CentOS', '7', { '1' => ['role1'], '2' => ['role2'] }, ['centos7-64role1.ma{hostname=centos7-64-1}-centos7-64role2.a{hostname=centos7-64-2}', 'CentOS 7']],
].each do |os, release, nodes, expected|
it { expect(described_class.os_release_to_setfile(os, release, nodes_and_roles: nodes)).to eq(expected) }
end
end

describe 'domain, puppet_version and nodes_and_roles' do
[
['CentOS', '7', 'mydomain.org', 'puppet7', { '1' => ['role1'], '2' => ['role2'] },
['centos7-64role1.ma{hostname=centos7-64-puppet7-1.mydomain.org}-centos7-64role2.a{hostname=centos7-64-puppet7-2.mydomain.org}', 'CentOS 7'],],
].each do |os, release, domain, puppet_version, nodes, expected|
it { expect(described_class.os_release_to_setfile(os, release, domain: domain, puppet_version: puppet_version, nodes_and_roles: nodes)).to eq(expected) }
end
end
end
end
Loading

0 comments on commit 670fbea

Please sign in to comment.