Role or playbook-focused collections: include in the Ansible or not? If yes, which criteria/requirements to use?

[Andersson007]

Summary

Was originally brought by @geerlingguy in ansible-collections/overview#127.

Several collections containing only roles have been recently submitted, so the topic is very relevant.

I can see the following questions at the moment that need to be answered:

Question 1: Are we generally OK with including role / playbook-focused collections in the Ansible package? Concerns?

Question 2: If the answer on Question 1 is positive, which criteria/requirements we should be used?

Any feedback would be much appreciated. After it's collected, we can vote.

[sean-m-sullivan]

I know Question 1 will likely be people on both sides, Where do we draw the line with it, do we make exceptions, For some reason I thought we had a standard already.

For Question 2, I think its largely around the current standards. Strong testing along with the other standards that are already in place.

[mariolenz]

My first reaction is to completely disallow roles in the Ansible package.

1. We have enough work already.
2. We would need to define quality criteria for roles, which sounds like a lot of work and... well, see above.
3. The community package is the successor to the old ansible, which didn't include any roles AFAIK (this is quite a weak argument, though).

I might be wrong, but those are my first thoughts about roles in Ansible / the community package.

[gotmax23]

I'm amendable to including these collections, but yeah, I'd like some basic standards/guidelines first.

[felixfontein]

@mariolenz we already have roles in the Ansible package, some collections include roles. Do you mean to forbid all roles, or just collections that have mostly / only roles?

In general, I think collections with roles, and also ones with mostly or even only roles should be OK. But we also have to consider the famous "nginx role problem", i.e. how many different nginx config roles would we want to have included (just search for 'nginx' on Galaxy)? I think we need some good consideration on which collections consisting mostly of roles to accept. Being more official, or more popular than other collections covering similar topics / products could be criteria to look for.

[mariolenz]

@mariolenz we already have roles in the Ansible package, some collections include roles. Do you mean to forbid all roles, or just collections that have mostly / only roles?

Yes, I've seen there are some collections including roles. My idea would be to forbid roles, otherwise we would need to have quality criteria. Or, otherwise, we we define we won't do any quality management for roles in collections. That would be a difference to modules in collections, where we try to do.

I think we need some good consideration on which collections consisting mostly of roles to accept. Being more official, or more popular than other collections covering similar topics / products could be criteria to look for.

My point exactly. Are we really able to define "acceptable" roles?

Just to make this clear, I'm not completely against roles in the community package. I just think it would be much easier for us if we disallow roles.

[felixfontein]

I'm against forbidding roles and playbooks completely - but then I'm also biased, since I added some (community.sops: install role and install on localhost playbook). For my specific case, I don't see any other (sane) way to solve this though - or do you want to force every user of community.sops to somehow install sops manually? (It is not available as an OS package in many cases.)

We did discuss this topic a bit at today's community meeting, and it quickly evolved into a discussion on what should be in Ansible in general, resp. what that package should be in general (i.e. basically #82). My personal opinion is that collections with mainly roles/playbooks, or even collections in general, should only be included if they are useful for a significant (whatever that means exactly) portion of the Ansible community package users.

[samccann]

@felixfontein how does your community.sops role example fit into that criteria - If I created a community.sops_roles_and_playbooks collection, would that be considered useful for a significant portion of package users?

[felixfontein]

@samccann a community.sops_roles_and_playbooks collection is probably not very useful (and should not be included). But if you want to use sops for managing secrets with Ansible, the plugins provided in community.sops are very useful (and this can be useful for a significant portion of package users IMO) - but to use them, you first have to install Mozilla sops itself. The easiest way for that is to use a role, since on many platforms sops is not available as a package for the system package manager, or (especially when creating EEs) a playbook (which installs sops on localhost and needs no further instructions). So from my POV having this installation role and playbook in community.sops is useful, while having other sops roles and playbooks in the same collection (or in other collections) is probably not useful.

[samccann]

@felixfontein ok that makes sense, thanks. It does complicate the inclusion criteria some, but it might work with the same voting mechanism - if enough 'people in the know' vote in favor of a particular foo.roles collection because it is helpful to many users etc. But I could also see it getting more contentious than the current voting for collection inclusions for example.

From the 30K foot view - seems to me roles/playbooks based collections should be considered for the Ansible package as it does extend the usability of it, if I understand things correctly. The package is 'batteries included' to make things easier for users, and to me, 'useful' roles and playbooks are part of those batteries - aka it's all content that helps the end user. But I as I said, it will get contentious on what is considered useful.

[russoz]

I think that we should really promote the use of ansible-galaxy collections install rather than cram them into the ansible package. I know this is another can of worms in itself, maybe we should raise a separate issue to discuss that.

Within the existing rules and standards, though, I am against forbidding that type of collection.

[Andersson007]

I think that we should really promote the use of ansible-galaxy collections install rather than cram them into the ansible package. I know this is another can of worms in itself, maybe we should raise a separate issue to discuss that.

Within the existing rules and standards, though, I am against forbidding that type of collection.

Yeah, otherwise we can end up having whole Galaxy included in the Ansible package which will be literally all the batteries included..

[briantist]

Question 1: Are we generally OK with including role / playbook-focused collections in the Ansible package? Concerns?

Yes. The package includes collections, roles and playbooks are part of collections, and I see no good reason to exclude them.

Question 2: If the answer on Question 1 is positive, which criteria/requirements we should be used?

This is much harder to answer admittedly, however I feel strongly that our inability to easily define such criteria is not a good reason to exclude these.

We can start with requiring a few things:

• roles - a role argument spec for all entrypoints that is accurate and works for both argument validation and documentation (which is more strict than the the argument validation requirements)
• roles & playbooks - some amount of linting (ansible-lint?)
• roles & playbooks - some amount of integration tests (ansible-test runme.sh, molecule, monkeyable)

Optional:

• roles & playbooks - unit-like tests (monkeyable)

For those who haven't seen it, monkeyable seems pretty nice for this because it does allow for some testing that is hard to do with any other tool (mocks).

---

We don't have to fully define everything right away, we can make changes over time, and in response to the roles and playbooks we see submitted.

Not sure on the following, or its wording but:
We can define in the inclusion criteria that role/playbook acceptance is currently under a rapidly changing provisional review, and that requirements may shift more quickly than our criteria for other types of content, and that contributors may need to be more responsive to these changes.

[felixfontein]

Yes, documentation is pretty important. Which means for collections included in Ansible, that the documentation can be read on docs.ansible.com. For roles, this requires a role argument spec, and for playbooks docs/docsite/extra.yml needs to be used. (Which should also be used for other documentation, like scenario guides.)

[MarkusTeufelberger]

A few issues with roles compared to modules/action plugins are:

• Documentation is largely not automated or standardized (yet) especially on the task level.
• Testing tools for Ansible "code" are lacking in comparison to tools dealing with Python code (code coverage(!), unit testing...).
• Idempotence is unclear for roles. Modules have very few success/failure states compared to roles that could return/fail on every single task along the way in theory. Not every role is written in a way (or even: can be written in a way!) that an intermittent failure just needs a rerun from the start.
• Choice/defaults vs. ease of use. A serious nginx role for example would have dozens if not hundreds of available variables and settings, similar to a heavily templated helm chart, and it would be near impossible to understand at a glance what it is doing (there would be more {{ }} than anything else in the tasks files most likely).
• Larger moving targets. Where modules usually deal with one piece of software, roles often deal with whole types of software (e.g. all package managers on all distros out there) and whole changing ecosystems (Debian, Red Hat, Arch, Alpine...). Modules already run into problems from time to time when it comes to staying on top of things while supporting older versions of dependencies still in use. Roles (especially when depending on modules from collections instead of just builtins) can and will have an even harder time with their dependencies, both ansible wise and within their target systems.
• The result of roles can at the moment not really be expressed and verified (other than writing a few assertions after its main work is done or along the way). Similar to the argument_spec.yml it probably would also be helpful if roles contained a verify.yml playbook/task list that contains all the expected changes on a system once the role was applied. Molecule for example does something similar to this.
• Already modules have no real performance targets as far as I'm aware to be included into ansible. Roles are even more sensitive to bad style when it comes to execution speed. It might make sense to also require that basic integration test scenarios finish within a certain time or at least that the duration of such tests is being recorded and displayed so users know what they are getting themselves into.

One main question is probably:

If roles are shipped with pip install ansible, should these be as trusted and flexible as modules (but also as comparatively hard to read - just like not every user of Ansible knows Python) or should these be in a style that users can understand at a glance and adapt to their needs if necessary? If a role is not supposed to be understood and modified by a user, it can be re-implemented as an action plugin relatively easy by the way. Then it will be hidden away from the user during execution, but it will be able to return "ok/changed/failed/skipped/..." instead of spewing its implementation all over the logs even though the user is not supposed to understand or care about any of it anyways.

I read through the community.sops.install role and it is a good example for a well written role that works on several Linux platforms - but also at the cost of containing ~30 tasks + 5 parameters for what is essentially curl ...sops.deb -o /tmp/sops.deb && apt install /tmp/sops.deb with a version selector. Does a "install a self contained program from upstream or github"-role really need to be this complex? In the general case: Yes - and if you want to support all possible options (Windows, installing the executable from github in /usr/bin/sops on distros without upstream packages that don't support debs or rpms...) it would get even more complex with no immediate benefit to e.g. a normal Debian user. At a certain point that user is going to look at half a dozen task files with several dozen tasks and replacing all of it with a single call to ansible.builtin.apt and then complaining about these "bloated and

[felixfontein]

@MarkusTeufelberger btw, your message got truncated :)

If roles are shipped with pip install ansible, should these be as trusted and flexible as modules (but also as comparatively hard to read - just like not every user of Ansible knows Python) or should these be in a style that users can understand at a glance and adapt to their needs if necessary?

That's a very good question. I'm not sure there is a good way to answer it.

If a role is not supposed to be understood and modified by a user, it can be re-implemented as an action plugin relatively easy by the way. Then it will be hidden away from the user during execution, but it will be able to return "ok/changed/failed/skipped/..." instead of spewing its implementation all over the logs even though the user is not supposed to understand or care about any of it anyways.

That is true, but it makes it even harder to read/understand/modify. (Though I guess that with a bit of boilerplating, it should be easier to read/modify than a role for anyone with some Python understanding - especially if it needs branching and reacting to a combination of user input and gathered facts, like community.sops.install does right now.)

Maybe we should try out to create some boilerplating utils that make it easy to create action plugins that encapsulate roles, and see how that goes - and then decide whether to push collections (at least the ones included in Ansible) who want to use roles to go this route, or to use real roles. Actually, the longer I think about this the more I want to try it out...

[MarkusTeufelberger]

Oof, too much of a rant before going to bed I guess. :-D

Here's a relatively complex role as an action plugin that we use for years already (and that needs a bit of an update probably soonish): https://github.com/mgit-at/ansible-collection-mgssl/blob/master/plugins/action/certificate.py

[MarkusTeufelberger]

In general I think the message still comes across (and I don't think I went on for much longer there) - roles are hard to deal with in Ansible - maybe harder than necessary?

Adding them to the community distribution might either give them a boost and cause some tooling etc. to spring into action to improve the situation or it might expose the weaknesses further and turn people away from them.

Personally I'd like to see a bit more feature completeness in e.g. https://github.com/ansible-community/molecule and a few well executed examples using it before opening the flood gates of nginx roles into ansible and controling rather tightly what gets entered until then (e.g. needs argument specs, needs to be written using exclusively builtins and modules from the same collection, needs to be testable and tested in official docker images (https://hub.docker.com/search?image_filter=official&q=) of at least most major Linux distros...) and then relax the requirements as the tooling and infrastructure around it catches up.

"Hiding" simpler roles in action plugins helps in one way to make roles seem like modules and getting the immediate benefits of the tooling around these, on the other hand it keeps the ecosystem from evolving and building much needed tools, standards, knowledge and infrastructure around writing, using and maintaining reusable and generally useful roles.

[briantist]

@MarkusTeufelberger most of the issues you've brought up are about roles themselves, and while I don't disagree about the points necessarily, I feel they don't really change anything in regards to including roles in the package.

For example, the balance of complexity vs. usefulness is an age-old issue in roles: do you make a small, tightly scoped role that does a single thing, for what might be a single platform and configuration set, or do you make your role try to be the "one true role" to do every single thing and take 100+ options, and a nest of conditionals and tasks?

It can be hard to strike a balance there, and it's one of the reasons I think that publishing roles in general is often not that useful, but the community disagrees.

However, the important thing is that it doesn't matter (imo) when it comes to deciding whether to include them, because we aren't really deciding when it comes to individual roles.

We accept collections, each of which must have one or more maintainers, and we trust them to make good decisions about what's important for a given collection and that collection's community.

Our role (heh) is to try to and define some level of requirements that we think is a proxy for quality, or really, maintainability, so that we have a way to communicate with collection maintainers about what we think is important, and in the worst case, remove the collection as a whole from the package.

---

I do agree that we don't have such mature examples or tooling around roles right now, and that is a hard problem, but there are options.

What I'd like to avoid is being too forceful with our own particular opinions on what roles should be and how they should be written and scoped, and leave it to communities who create and use them. In short, I don't want to micromanage roles.

By that I mean, I don't think it's our place to define performance characteristics, or specific platforms that must be supported, etc.

---

I also wanted to address your comments about roles being understandable. This is a nice thing even just for maintainers and contributors, and it's a nice-to-have for end users when it comes to being able to submit bug reports or feature requests, but I ultimately don't think it's a problem by itself that a role is very complicated or difficult to understand: this is kind of the point of these being created and published.

We don't expect end users to understand how a module works, and we don't expect them to be making local modifications to them, and the same is true for roles. There isn't even a supported mechanism for doing so.

Documentation on how to use them is important, and I'd argue that contrary to how the roles you write and use for yourself or your company or team are written, published roles should focus on being easy to use which often comes at the expense of being more complex in implementation. Anyway now I'm getting into an opinion, which again isn't something I want to enforce, just giving a POV that I don't think we need to be overly concerned with end-users of a role being able to grok it.

---

Re: converting roles to action plugins: I don't think most roles lend themselves well to this tbh. They can become real cumbersome real quick, and it ends up hiding a bunch of stuff that's happening underneath. I imagine debugging them to be particularly annoying, unless there's a bunch of error handling that ends up just re-implementing what ansible would do when executing plays.

I wouldn't stop anyone from doing it, but I don't think I'd go that direction myself except for very simple "roles" that have a good reason for it.

---

Last thing I want to address is a more general thing that many folks seem to be raising, which is the idea that the "flood-gates are opening" and there's this concern that the package is going to become overrun with roles or role-based collections.

I'm trying to understand where this worry keeps coming from, I don't really see it happening, and I wonder a bit why we seem to be so collectively worried about it? Am I missing something?

[pyrodie18]

Going to try and bring this conversation back up and get more input. The community.zabbix collection includes a number of roles that we've maintained over the years. Currently the roles support 3 versions of Zabbix across something like 9 OS/Versions. Anything we support includes full molecule testing. We have "best effort" supported a couple of other OSs but couldn't readily test for them. We have a pending PR and looks to expand the OSs being covered but can't fully be tested in some ways (ansible-collections/community.zabbix#1194) My question is in order to "support" something, does that require testing. If something is only best effort and that test fails is that sufficient to stop the acceptance of a PR.

[felixfontein]

In my opinion, for everything that's community maintained/"supported" and where you aren't selling support contracts, making sure that things work is always best effort, and there never is a guarantee that it works in every and all cases, and doesn't break random things. (If someone wants more guarantees, they have to pay you or someone else to ensure them.)

So to answer your question: I would try to test as much as possible that can be tested with a reasonable amount of work, and try not to implement too much functionality that cannot be properly tested. But having some things implemented that cannot be fully tested is totally OK in my opinion.

[sean-m-sullivan]

Also on the other things, are the infra roles, to manage AAP itself, we stand them up test them. I believe that as long as the collections are maintained, have CI covering a majority of the roles in the collection, they should be eligible for inclusion, just like any other community based module only collection.

[mariolenz]

@Andersson007 Please close this issue if done, or open a new forum topic and then close the issue with a pointer to the new discussion: Community-topics: Archiving the repo

[Andersson007]

@mariolenz thanks for pinging!

I'll close this one. Whoever else first wants to continue discussing this, please open a forum topic referring to this issue.