[RAC][Rule Registry] Namespaces that are prefixes of other namespaces can cause unexpected behaviors

[marshallmain]

Background
Currently the RAC index naming scheme is .alerts-{registrationContext}.{datasetSuffix}-{namespace}, where {namespace} is a user defined string that allows admins to arbitrarily segment alerts by index. The security solution will store space IDs in {namespace} to provide continuity with the existing .siem-signals-<space id> index naming scheme.

After we update the rule registry to add the necessary index template per namespace (draft PR) the index pattern defined in the index template generated for a particular index alias will be .alerts-{registrationContext}.{datasetSuffix}-{namespace}-* so that the index template will be applied to all concrete indices that are created for the alias, e.g. .alerts-{registrationContext}.{datasetSuffix}-{namespace}-000001.

Namespace conflicts
The problem is that the trailing * on the index pattern means namespace pairs where one is a prefix of the other will both match the same pattern. For example, if 2 namespaces were managers and managers-usa, then the index patterns would be .alerts-security.alerts-managers-* and .alerts-security.alerts-managers-usa-*. Concrete indices for the managers-usa namespace would match both patterns and without setting a priority on the index templates there's no way to pick which template to apply. Elasticsearch disallows even creating index templates with the same priority where the patterns overlap like this.

There are a number of ways we could choose to address this problem:

1. Choose a delimiter between index name components that is not allowed to be in the namespace string. It may be difficult to find a suitable delimiter if there are no special characters that are usable in index names but not space ids.
2. Move the {namespace} away from the end of the index name. We control the {datasetSuffix} and will most likely never include dashes in the suffix itself, since we use dashes as delimiters. This would prevent the issue above where a single index name could match multiple index templates. However, including the user defined string (which might include dashes) in the middle of the index name might introduce other bugs or even security holes if we attempt to parse the index name without being very aware of all edge cases introduced by the user defined string in the middle.
3. Document the potential pitfalls with namespaces where one namespace is a prefix of another and tell users not to do that.
4. Resolve the index template ambiguity by setting template priorities. It may be possible to set the index template priority based on the length of the namespace string so that if an index matches both templates then it chooses the one with the longer namespace.

We should keep an eye out for other areas where we use a trailing -* on index patterns, since with the user defined {namespace} on the end of the index it's very possible for this sort of pattern to lead to unexpected behaviors.

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[marshallmain]

Fixed by the combination of #108115 (which sets index template priority, so a longer namespace has higher priority) and #107991 (retrieving existing indices using a combination of the alias and backing index pattern, rather than just the backing index pattern)