Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert ensure_packages to new API and refactor #1244

Merged
merged 2 commits into from
Jul 11, 2022

Conversation

alexjfisher
Copy link
Collaborator

As a first step before fixing #1196 in a separate PR, this PR converts to the new API and gets rid of some code duplication etc. in the process.

@alexjfisher alexjfisher requested a review from a team as a code owner May 20, 2022 16:13
@puppet-community-rangefinder
Copy link

ensure_packages is a function

Breaking changes to this file MAY impact these 527 modules (near match):

ensure_packages is a function

Breaking changes to this file MAY impact these 527 modules (near match):

This module is declared in 318 of 579 indexed public Puppetfiles.


These results were generated with Rangefinder, a tool that helps predict the downstream impact of breaking changes to elements used in Puppet modules. You can run this on the command line to get a full report.

Exact matches are those that we can positively identify via namespace and the declaring modules' metadata. Non-namespaced items, such as Puppet 3.x functions, will always be reported as near matches only.

REFERENCE.md Outdated Show resolved Hide resolved
lib/puppet/functions/ensure_packages.rb Outdated Show resolved Hide resolved
ekohl
ekohl previously approved these changes Jun 9, 2022
lib/puppet/functions/ensure_packages.rb Outdated Show resolved Hide resolved
lib/puppet/functions/ensure_packages.rb Outdated Show resolved Hide resolved
Copy link
Collaborator

@bastelfreak bastelfreak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, but someone else should take a look as well.

ekohl
ekohl previously approved these changes Jun 9, 2022
Copy link
Collaborator

@ekohl ekohl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code wise 👍 just some documentation suggestions. I also wonder if @see ensure_resource could be added, but I'm not 100% sure puppet-strings properly supports that.

@@ -0,0 +1,38 @@
# frozen_string_literal: true

# @summary Takes a list of packages and only installs them if they don't already exist.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically you can also ensure they're absent via default_attributes, right?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, or by specifying a hash for the first parameter. But... I really don't know why people do that. IMO it becomes a fairly useless function the moment you start specifying attributes as they likely won't match with any other definition you already have in the catalog for that package.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps manages instead of installs is better? Also, due to the behavior of ensure_resource(s) the state must match. So perhaps ensures is even better.

Suggested change
# @summary Takes a list of packages and only installs them if they don't already exist.
# @summary Takes a list of packages and ensures they exist

lib/puppet/functions/ensure_packages.rb Outdated Show resolved Hide resolved
@hlindberg
Copy link
Contributor

I suspect this function needs to be an InternalFunction (see function contain, require in puppet source how that is done. It also needs to capture the calling scope (like those functions do), and then make the call to ensure_resource using that scope. The risk is otherwise that a resource won't show up with the correct containment (where it came from) in case it was created by the call to ensure_packages. I think you need to add a test case for that as well, and I suspect it will show that scope must be passed on.

@ekohl
Copy link
Collaborator

ekohl commented Jun 9, 2022

Containment is an interesting point. I think today it only does something if the resource was not previously defined. Otherwise it may be contained (I don't know). So it's already ambiguous where the package resource ends up. I've always written my code by assuming it was not contained added explicit require properties if needed.

You could add in containment, but I wonder if that could cause risks with dependency cycles. I'm thinking about this because the package may not be contained within two classes.

@alexjfisher
Copy link
Collaborator Author

Good call. I think it was contained, by the calling class and now it isn't.

@alexjfisher
Copy link
Collaborator Author

See https://gist.github.com/alexjfisher/b0287dc84a5ce451093d43f2de16f4ed

I'll look into making this an Internal function to preserve the existing behaviour. Not sure how to write a test case for this. @hlindberg any pointers? Do I just do that_comes_before('Whit[Completed_class[Foo]]') ???

@hlindberg
Copy link
Contributor

@alexjfisher sorry, don't know how to write that test.

@alexjfisher
Copy link
Collaborator Author

@alexjfisher sorry, don't know how to write that test.

No luck so far. The meta whit resources don't even appear to be in the catalogue object rspec-puppet generates.

@hlindberg
Copy link
Contributor

I also think the function to be called (with the scope passed) needs to be called a special way - IIRC, there should be something like call_function_with_scope available to the internal-function. Unfortunately, the contain and require functions in puppe where optimized to instead of calling include they now do what include did as it was faster - so you cannot immediately see how it can be done. You may be able to look at the history of those functions to see how they used to make "call include with given scope".

@alexjfisher
Copy link
Collaborator Author

Not just scope.call_function('ensure_resource', ['package', package_name, defaults]) instead of call_function('ensure_resource', 'package', package_name, defaults) ?

I'll need to check on Monday morning, but I think that's what my uncommitted change I was working on looks like. Think it works. Need to double check.

To preserve original behaviour of the created package(s) being contained
within the class that called `ensure_packages`, the function needs to be
an `InternalFunction` and call `ensure_resource(s)` with the correct
scope.
@alexjfisher
Copy link
Collaborator Author

@hlindberg Does b565f7f look 'correct' to you? In my testing with the --graph option, this 4x function now produces an identical graph to the 3x function it replaces.

@binford2k
Copy link
Contributor

I'm going to vote pretty hard that we do not try to maintain containment artifacts. At best, the existing behavior is unspecified and depending on its implementation details already causes surprising failures.

Given the link that @alexjfisher posted earlier, the graph will already be different if someone includes the tzdata package somewhere else, either explicitly or with ensure_packages() and it happens to be declared first.

Imho, that's the whole point of this function. It explicitly says "I don't care how this package gets declared, just make sure it's in the catalog somewhere."

@ekohl
Copy link
Collaborator

ekohl commented Jun 14, 2022

I do agree with @binford2k, with the note that changing it in a minor/patch version might be surprising. Breaking implicit behavior still is painful for users.

@alexjfisher
Copy link
Collaborator Author

alexjfisher commented Jun 14, 2022

Thanks for the feedback.

To summarise...

  • I'll drop b565f7f
  • Mark this a breaking change
  • Update the first comment with a more user friendly summary of why it might be breaking in some cases

Once merged, I can then get on with coming up with a fix for making it work no matter if any existing package resource has ensure => present, or ensure => installed.

@ekohl
Copy link
Collaborator

ekohl commented Jun 14, 2022

I'd be happy with merging it as is, releasing it in a minor version and then doing the breaking change later. Major version bumps in stdlib are painful so I'd rather combine a few things.

For example, perhaps we should deprecate the compat data types and drop them in the next major version (looking at you https://github.com/puppetlabs/puppetlabs-stdlib/tree/main/types/compat).

@chelnak
Copy link
Contributor

chelnak commented Jul 11, 2022

@ekohl are you still good for this change to be merged?

Copy link
Collaborator

@ekohl ekohl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chelnak yes, let's get this in.

@ekohl ekohl merged commit ead5c6b into puppetlabs:main Jul 11, 2022
@chelnak
Copy link
Contributor

chelnak commented Jul 11, 2022

@ekohl Great thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants