Output of migration system_features incorrectly reports all dot prefixed indices as system indices

[pgomulka]

Elasticsearch Version

7.17

Installed Plugins

No response

Java Version

bundled

OS Version

mac

Problem Description

Kibana is using /_migration/system_features to filter out indices that require a migration from _migration/deprecations?pretty output.
while it makes sense for 7.7+ created indices (the validation to prevent use a . dot prefixed index name) it will prevent to do a correct upgrade from 7.x to 8.x as it is still ok to have 6.x or <7.7 created indices.

We should change the logic in /_migration/system_features to take into account that an index created in <7.7 migth still have a . in a prefix

a validation of dot prefixed index names #49959

Steps to Reproduce

1. in <7.7 create a dot prefixed index name
2. upgrade to 7.7+
3. run a migration api /_migration/system_features
   observe that an index is reported as a system index

Logs (if relevant)

No response

[elasticsearchmachine]

Pinging @elastic/es-core-infra (Team:Core/Infra)

[droberts195]

Output of migration system_features incorrectly reports all dot prefixed indices as system indices

I am not convinced that it really is all.

Certainly for ML, the ML internal indices are divided between system indices and internal hidden indices. The system indices are supposed to be invisible to users, and upgraded automatically by the feature migration functionality, while the hidden indices are intended to be searchable by end users, and if they need reindexing before a major version upgrade then this is handled by the Kibana migration assistant.

The relevant line of code seems to be:

elasticsearch/server/src/main/java/org/elasticsearch/action/admin/cluster/migration/TransportGetFeatureUpgradeStatusAction.java

Line 164 in 602ccd8

.flatMap(descriptor -> descriptor.getMatchingIndices(state.metadata()).stream())

If the problem really was as broad as the current issue description suggests I would have expected to have had bug reports against ML by now.

I am wondering if the real problem is, "Output of migration system_features incorrectly reports all indices that match patterns in system index descriptors as system indices even if they are very old and aren't covered by the functionality that the pattern was intended to apply to"?

For example, take:

elasticsearch/modules/kibana/src/main/java/org/elasticsearch/kibana/KibanaPlugin.java

Line 34 in 0ddb590

.setIndexPattern(".reporting-*")

If there are ancient indices that match .reporting-* but don't get migrated automatically like a modern reporting index would then that would be a problem covered by this bug. But an ancient .ml-anomalies-custom-my_really_old_job index wouldn't match any system index descriptor, so should just be left to the Kibana migration assistant to reindex as necessary.

I want to make sure this bug is scoped correctly because I can imagine that a proposed fix based on an incorrect view of what's affected might cause more problems for features that knowingly use a mix of dot-prefixed system and hidden indices (like ML).

[pgomulka]

"Output of migration system_features incorrectly reports all indices that match patterns in system index descriptors as system indices even if they are very old and aren't covered by the functionality that the pattern was intended to apply to"
I think you are right. This is probably a marginal problem.
.reporting was created before 6.2 (when .reporting started to be used by kibana)
I will discuss this with core/infra anyway to see if we should do any action.