[Remote Store] Update remote Store flows to support any path type with backward compatibility

[ashking94]

Is your feature request related to a problem? Please describe

With optimised prefix pattern for remote store path (as mentioned in #12567), we need to ensure that we are able to allow new indexes to use the optimised remote store path type and at the same time be able to resolve the existing prefix pattern (Fixed remote store path type).

Describe the solution you'd like

As part of this task, we intend to make the necessary code changes to allow new indexes to use any remote store path type based on the cluster setting cluster.remote_store.index.path.prefix.type. We have added a remote store custom data in index metadata that is used to associate the path type for an index. When the custom data is absent, we would treat the path type to be Fixed.

Related component

Storage:Remote

Describe alternatives you've considered

No response

Additional context

No response