Audit features in tech-preview and remove the lables where appropriate

[ipanova]

Audit features in tech-preview and remove the label where appropriate

[bmbouter]

If a summary of the labels can be posted that would be helpful.

One goal of the analytics.pulpproject.org work is to have analytics tell us if feature X was being used or not. For example: alternate content sources is in tech preview, but it would be good to know if the pulp_rpm or pulp_file (the two plugins that implement ACS) have users actually configuring an ACS. I believe if very few or zero users are using an ACS then we should leave it in tech preview. That data can be combined with data of any kind we have on the usage of tech preview labels.

[ipanova]

@bmbouter I can see the benefit of the analytics goal, however, I don't think we should rely on its results which can lead to have the feature in tech preview for an indefinite time.
Tech preview labels should be removed as soon as possible. Here are my thoughts on why:

1. Even though ANALYTICS are enabled by default, users can disable that, therefore the results on the feature usage might not be accurate. In addition, a feature in tech preview means it is not fully supported, may not be functionally complete, and is not suitable for deployment in production. Tech preview feature may also be dropped from future release. This generally leads the feature not being used in production because of fear of instability and incompatible changes.
2. The goal to have a feature in tech preview is to a) show that it is under active development and testing but can lead to backwards incompatible changes b) gain wider exposure and gather as much feedback as possible from the users who can influence its development. While these points are no longer true: no development, testing or feedback received, I don't see why keeping the feature in tech preview for N releases. The absence of feedback is painful one, but to me, it does not make sense to keep a feature in tech preview in a hope that one day someone will start using it and tell us something about it. We whether decide to support it and remove it from tech preview or drop it completely.

[bmbouter]

What's wrong with leaving them in tech preview? If we have no feedback on them being used (analytics or otherwise) then to me this isn't a priority.

[ipanova]

Tech preview label can have negative connotation for the mentioned reasons in my previous comment - it's like fedora rawhide you most likely won't install in production and wait for the official release instead.

Besides that point, sure, it is not a priority but leaving this lying around for months when no activity is ongoing also does not sound right.

[bmbouter]

A list of all the tech preview features and API endpoint (or mgmt commands maybe if they are there too) would be helpful. Maybe that's what this ticket is for because each would have to be done with discussion.

[ipanova]

As the author of this ticket my intention was to identify a list of all features and functionality that is in tech preview, remove the labels from those that are not under active development anymore. Prompted discussion would happen on PR itself.
Maybe current situation of this list is not debatable, what mostly interests me, and that's a separate thread, is definition of our future actions with regards to how we treat tech preview labels and its respective features.

[bmbouter]

Going forward (and for our existing ones too) I'm hoping for data driven decision making. I'd like to see data from actual users. For example when Katello takes a tech preview label feature like ACS and integrates it into Katello, the actual data would come from when that version is released, users have installed it, and we know users are actually enabling it in Katello. Knowing all that has happened is pretty hard, which is what brings me continually back to analytics.

[lubosmj]

It is also worth auditing plugins. We still consider some of the RBAC features to be in tech preview. This is no longer true.

Features "still" in tech preview:

• Pulp Labels - recent changes made to the database model; used by a community, stable interface; I, personally, would leave it in tech preview.

Features no longer in tech preview:

• Caching responses by leveraging Redis (CACHE_ENABLED) - used by plugins, sporadic bug-fixing
• Content signing (pulpcore-manager [remove|add]-signing-service) - used by plugins and users
• ACS - used by users
• Import/Export - used by users
• Filesystem exporters - no significant development observed; one BZ opened
• RBAC (this will have to be revisited in respective plugins) - used heavily by pulp_container
• Handling artifact checksums (handle-artifact-checksums --checksums --report) - used by users
• Plugin removal (pulpcore-manager remove-plugin) - no significant development recorded
• Retaining repository versions - used by users
• Reclaiming disk space - used by users, bug-fixing only
• Task scheduling - no changes since the last year; not sure about the user background
• Pulp self-repair - no recent significant changes to the workflow

Features that could be eliminated:

• Stages profiling (PROFILE_STAGES_API) - prone to errors, sporadic bug-fixing

I have noticed that we also state in comments all around the code that some of the features are still in tech preview. This might not be even visible to end-users:

• TasksViewSet.purge - used by users; should not be in tech preview
• [User|Group|...]ViewSet - used by users; yet, we still make changes to the API from time to time; should be in tech preview

[ipanova]

[User|Group|...]ViewSet - used by users; yet, we still make changes to the API from time to time; should be in tech preview

Given that this functionality was introduced over 2 years ago I think it is safe to remove the preview flag 7a11ac4

TasksViewSet.purge - used by users; should not be in tech preview

And this one over a year ago d6f5d00

[bmbouter]

Of the list of "Features no longer in tech preview", as I understand it, they are almost all still in tech preview. For example my understanding is ACS is still in tech preview.

Yes some things have been released for a long time, however, couldn't it be that those features are just unused? Have we received user feedback on those items?

[ipanova]

Of the list of "Features no longer in tech preview", as I understand it, they are almost all still in tech preview. For example my understanding is ACS is still in tech preview.

Yes some things have been released for a long time, however, couldn't it be that those features are just unused? Have we received user feedback on those items?

It could, but in that case we should whether remove that feature completely or just commit officially to support as is. There have not been any development for X long time what are the chances there will be any?

[lubosmj]

We are still receiving some useful feedback from https://discourse.pulpproject.org/t/features-in-tech-preview/737/5. Hold on tight!