(MODULES-10873) Add support for puppet7 collection #524

joshcooper · 2020-11-19T20:32:33Z

This makes it possible to install puppet-agent from the puppet7 collection,
while keeping the current behavior of defaulting to puppet6.

Bump the github action to ruby 2.5 as puppet7 doesn't support earlier versions.

Add --trace argument to puppet apply run that performs the upgrade.

puppet-community-rangefinder · 2020-11-19T20:32:44Z

puppet_agent::install::solaris is a class

that may have no external impact to Forge modules.

puppet_agent::service is a class

that may have no external impact to Forge modules.

This module is declared in 3 of 575 indexed public `Puppetfiles`.

These results were generated with Rangefinder, a tool that helps predict the downstream impact of breaking changes to elements used in Puppet modules. You can run this on the command line to get a full report.

Exact matches are those that we can positively identify via namespace and the declaring modules' metadata. Non-namespaced items, such as Puppet 3.x functions, will always be reported as near matches only.

puppet-community-rangefinder · 2020-11-19T23:49:39Z

puppet_agent::install::solaris is a class

that may have no external impact to Forge modules.

puppet_agent::service is a class

that may have no external impact to Forge modules.

This module is declared in 3 of 575 indexed public `Puppetfiles`.

These results were generated with Rangefinder, a tool that helps predict the downstream impact of breaking changes to elements used in Puppet modules. You can run this on the command line to get a full report.

Exact matches are those that we can positively identify via namespace and the declaring modules' metadata. Non-namespaced items, such as Puppet 3.x functions, will always be reported as near matches only.

gimmyxd · 2020-11-20T13:07:02Z

Seems like the task does not want to upgrade ubuntu 18.04 nodes:

  12:55:01 +0000 INFO: Version parameter defined: latest
  12:55:01 +0000 INFO: Downloading Puppet latest for Ubuntu...
  12:55:01 +0000 INFO: Ubuntu platform! Lets get you a DEB...
  12:55:01 +0000 INFO: Downloading http://apt.puppet.com/puppet7-release-bionic.deb
  12:55:01 +0000 INFO:   to file /tmp/install.sh.1361.60499/puppet7-release-bionic.deb
  12:55:01 +0000 INFO: Trying wget...
  12:55:01 +0000 INFO: Installing puppetlabs apt repo with dpkg...
  12:55:01 +0000 INFO: Version 6.19.1 detected...
  12:55:01 +0000 INFO: Collection upgrade detected, replacing puppet6-release
  Selecting previously unselected package puppet7-release.
  Hit:1 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote bionic-security InRelease
  Hit:2 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote bionic-updates InRelease
  Hit:3 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote bionic InRelease
  Hit:4 http://apt.puppetlabs.com bionic InRelease
  Reading package lists...
  Reading package lists...
  Building dependency tree...
  Reading state information...
  puppet-agent is already the newest version (6.19.1-1bionic).
  0 upgraded, 0 newly installed, 0 to remove and 112 not upgraded.

gimmyxd · 2020-11-20T13:24:02Z

I think i figured it out what is happening:
- we are installing puppet-agent 6, in order to do that we download puppet6-release and install it, but this install puppet-release instead of puppet6-release
- we start the upgrade to puppet 7, but first we need to remove the old release package, which we determine by the major version of the puppet-agent installed package: https://github.com/puppetlabs/puppetlabs-puppet_agent/blob/main/tasks/install_shell.sh#L464-L472
- this will fail to remove the release package and the next apt install -y puppet-agent will be a noop

host:~# dpkg -i puppet6-release-bionic.deb
Selecting previously unselected package puppet-release.
(Reading database ... 86341 files and directories currently installed.)
Preparing to unpack puppet6-release-bionic.deb ...
Unpacking puppet-release (1.0.0-14bionic) ...
Setting up puppet-release (1.0.0-14bionic) ...

host:~# dpkg -i --force-confmiss puppet7-release-bionic.deb
dpkg: regarding puppet7-release-bionic.deb containing puppet7-release:
 puppet7-release conflicts with puppet-release
  puppet-release (version 1.0.0-14bionic) is present and installed.

dpkg: error processing archive puppet7-release-bionic.deb (--install):
 conflicting packages - not installing puppet7-release
Errors were encountered while processing:
 puppet7-release-bionic.deb

docker/centos/Dockerfile.versions

docker/ubuntu/Dockerfile.versions

joshcooper · 2020-11-20T19:03:27Z

I verified I could upgrade from 6.19.1 to 7.0.0 on centos and ubuntu:

$ docker build -f docker/centos/Dockerfile . -t pa-6.19.1-dev:centos --build-arg before=6.19.1
...
$ docker run -it pa-6.19.1-dev:centos 7.0.0
...
Debug: Package[puppet-agent](provider=yum): Upgrading package puppet-agent from version 6.19.1-1.el7 to 7.0.0
Debug: Executing: '/usr/bin/yum -d 0 -e 0 -y update puppet-agent-7.0.0'
Debug: Executing: '/usr/bin/rpm -q puppet-agent --nosignature --nodigest --qf %{NAME} %|EPOCH?{%{EPOCH}}:{0}| %{VERSION} %{RELEASE} %{ARCH}\n'
Notice: /Stage[main]/Puppet_agent::Install/Package[puppet-agent]/ensure: ensure changed '6.19.1-1.el7' to '7.0.0'

$ docker build -f docker/ubuntu/Dockerfile . -t pa-6.19.1-dev:ubuntu --build-arg before=6.19.1
...
$ docker run -it pa-6.19.1-dev:ubuntu 7.0.0
...
Debug: Executing: '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold --force-yes install puppet-agent=7.0.0-1xenial'
Notice: /Stage[main]/Puppet_agent::Install/Package[puppet-agent]/ensure: ensure changed '6.19.1-1xenial' to '7.0.0-1xenial'

The upgrade from 5.5.21 completes but fails to save the report due code mismatch. Filed as https://tickets.puppetlabs.com/browse/MODULES-10875

joshcooper · 2020-11-20T20:17:33Z

 we start the upgrade to puppet 7, but first we need to remove the old release package, which we determine by the major version of the puppet-agent installed package: https://github.com/puppetlabs/puppetlabs-puppet_agent/blob/main/tasks/install_shell.sh#L464-L472

Thanks @gimmyxd that makes sense. If I change the line to dpkg --purge "puppet-release" then it correctly removes the old repo and upgrades to puppet7:

  20:10:50 +0000 INFO: Installing puppetlabs apt repo with dpkg...
  20:10:50 +0000 INFO: Version 6.19.1 detected...
  20:10:50 +0000 INFO: Collection upgrade detected, replacing puppet6-release
  (Reading database ... 84315 files and directories currently installed.)
  Removing puppet-release (1.0.0-14bionic) ...
  Purging configuration files for puppet-release (1.0.0-14bionic) ...
  (Reading database ... 84310 files and directories currently installed.)
  Preparing to unpack .../puppet7-release-bionic.deb ...
  Unpacking puppet7-release (7.0.0-1bionic) ...
  Setting up puppet7-release (7.0.0-1bionic) ...
  Hit:1 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote bionic-security InRelease
  Hit:2 http://apt.puppetlabs.com bionic InRelease
  Hit:3 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote bionic-updates InRelease
  Hit:4 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote bionic InRelease
  Get:5 http://apt.puppetlabs.com bionic/puppet7 amd64 Packages [2,414 B]
  Get:6 http://apt.puppetlabs.com bionic/puppet7 all Packages [1,725 B]
  Get:7 http://apt.puppetlabs.com bionic/puppet7 i386 Packages [1,725 B]
  Fetched 5,864 B in 1s (10.6 kB/s)
  Reading package lists...
  Reading package lists...
  Building dependency tree...
  Reading state information...
  The following packages will be upgraded:
    puppet-agent
  1 upgraded, 0 newly installed, 0 to remove and 112 not upgraded.
  Need to get 21.0 MB of archives.
  After this operation, 5,439 kB disk space will be freed.
  Get:1 http://apt.puppetlabs.com bionic/puppet7 amd64 puppet-agent amd64 7.0.0-1bionic [21.0 MB]
  Fetched 21.0 MB in 1s (40.3 MB/s)
(Reading database ... 84315 files and directories currently installed.)
  Preparing to unpack .../puppet-agent_7.0.0-1bionic_amd64.deb ...
  Unpacking puppet-agent (7.0.0-1bionic) over (6.19.1-1bionic) ...
  Setting up puppet-agent (7.0.0-1bionic) ...
  Processing triggers for libc-bin (2.27-3ubuntu1) ...

That said I'm thinking either the module should check if the package is actually installed prior to purging it, or somehow capture the fact that dpkg --purge didn't do anything. It doesn't seem we can rely on the exit code:

# dpkg --purge doesntexist
dpkg: warning: ignoring request to remove doesntexist which isn't installed
# echo $?
0

Is there a flag we can pass to dpkg --purge so the command fails if the package isn't installed?

Also note this PR should pass once we resolve the release packages for bionic (and buster and el8).

joshcooper · 2020-11-20T20:56:01Z

Is there a flag we can pass to dpkg --purge so the command fails if the package isn't installed?

Ah, it looks like the code assumes if $installed_version is set (which is done by cat /opt/puppetlabs/puppet/VERSION, then the corresponding release package named puppet${major}-release must be installed. And normally it should be. We could add a more defensive check on all platforms, but I think that is a separate ticket.

joshcooper · 2020-11-20T22:13:57Z

bionic packages were updated, should be good to go now

tasks/install_shell.sh

This makes it possible to install puppet-agent from the puppet7 collection, while keeping the current behavior of defaulting to puppet6. Bump the github action to ruby 2.5 as puppet7 doesn't support earlier versions. Add `--trace` argument to puppet apply run that performs the upgrade.

Dorin-Pleava

Tested with PE on debian and centos agents, and works as expected

joshcooper requested a review from a team as a code owner November 19, 2020 20:32

joshcooper requested a review from a team November 19, 2020 20:32

joshcooper closed this Nov 19, 2020

joshcooper reopened this Nov 19, 2020

joshcooper force-pushed the puppet7 branch from 3f41eba to a236246 Compare November 20, 2020 00:08

joshcooper commented Nov 20, 2020

View reviewed changes

docker/centos/Dockerfile.versions Outdated Show resolved Hide resolved

joshcooper commented Nov 20, 2020

View reviewed changes

docker/ubuntu/Dockerfile.versions Outdated Show resolved Hide resolved

joshcooper force-pushed the puppet7 branch from a236246 to 660ffb5 Compare November 20, 2020 18:55

DavidS mentioned this pull request Nov 23, 2020

Buildevents v2 puppetlabs/puppetlabs-testing#321

Merged

gimmyxd approved these changes Nov 24, 2020

View reviewed changes

gimmyxd requested a review from a team November 24, 2020 10:06

GabrielNagy reviewed Nov 24, 2020

View reviewed changes

tasks/install_shell.sh Outdated Show resolved Hide resolved

joshcooper force-pushed the puppet7 branch from 660ffb5 to f7264f4 Compare November 30, 2020 18:47

Dorin-Pleava approved these changes Dec 2, 2020

View reviewed changes

Dorin-Pleava merged commit be06195 into puppetlabs:main Dec 2, 2020

joshcooper deleted the puppet7 branch December 3, 2020 18:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

(MODULES-10873) Add support for puppet7 collection #524

(MODULES-10873) Add support for puppet7 collection #524

joshcooper commented Nov 19, 2020 •

edited

Loading

puppet-community-rangefinder bot commented Nov 19, 2020

puppet-community-rangefinder bot commented Nov 19, 2020

gimmyxd commented Nov 20, 2020

gimmyxd commented Nov 20, 2020

joshcooper commented Nov 20, 2020

joshcooper commented Nov 20, 2020 •

edited

Loading

joshcooper commented Nov 20, 2020

joshcooper commented Nov 20, 2020

Dorin-Pleava left a comment

(MODULES-10873) Add support for puppet7 collection #524

(MODULES-10873) Add support for puppet7 collection #524

Conversation

joshcooper commented Nov 19, 2020 • edited Loading

puppet-community-rangefinder bot commented Nov 19, 2020

puppet_agent::install::solaris is a class

puppet_agent::service is a class

This module is declared in 3 of 575 indexed public Puppetfiles.

puppet-community-rangefinder bot commented Nov 19, 2020

puppet_agent::install::solaris is a class

puppet_agent::service is a class

This module is declared in 3 of 575 indexed public Puppetfiles.

gimmyxd commented Nov 20, 2020

gimmyxd commented Nov 20, 2020

joshcooper commented Nov 20, 2020

joshcooper commented Nov 20, 2020 • edited Loading

joshcooper commented Nov 20, 2020

joshcooper commented Nov 20, 2020

Dorin-Pleava left a comment

Choose a reason for hiding this comment

joshcooper commented Nov 19, 2020 •

edited

Loading

This module is declared in 3 of 575 indexed public `Puppetfiles`.

This module is declared in 3 of 575 indexed public `Puppetfiles`.

joshcooper commented Nov 20, 2020 •

edited

Loading