Add support for closed indices to ILM

[dakrone]

Now that closed indices are replicated (see: #39499 / #33888). We should add support
for these indices to ILM.

This means that in a policy, a user should be able to specify that an index be
closed, and then still be able to perform some subset of operations on the
index (allocation routing, likely?).

The closed index should be allowed to be deleted in the delete phase of ILM as
well.

[elasticmachine]

Pinging @elastic/es-core-features

[jakelandis]

I think there are two areas of support here:

a user should be able to specify that an index be closed

It seems that now the we have frozen indices the reasons one would want to close an index are diminishing. By offering close AND frozen as options are we asking the user to make a nuanced decision ? I suspect the most users would pick close since it seems obvious what it does, and thus we may steer people away from frozen and it's benifits. (Users can always close the index themselves, but does it make sense to do so automated ?)

The closed index should be allowed to be deleted in the delete phase

Same argument as above, given that we have frozen, does close belong in an automated workflow ?

@eskibars - thoughts ?

[dakrone]

Same argument as above, given that we have frozen, does close belong in an automated workflow ?

My personal opinion here is yes, for instance, I can imagine that it would be nice to freeze older indices while still having them be searchable, and then, close them the day before the indices are deleted. This would allow anyone consuming the data to notice the data was missing (say, in a Kibana dashboard), and then take steps to save the data prior to it being deleted for good.

Definitely good to discuss it though.

[eskibars]

By offering close AND frozen as options are we asking the user to make a nuanced decision ? I suspect the most users would pick close since it seems obvious what it does, and thus we may steer people away from frozen and it's benifits.

This is my concern as well.

I can imagine that it would be nice to freeze older indices while still having them be searchable, and then, close them the day before the indices are deleted. This would allow anyone consuming the data to notice the data was missing (say, in a Kibana dashboard), and then take steps to save the data prior to it being deleted for good.

I suspect it'd be a mix with users. For smaller orgs/projects, the consumers of the data may also be the cluster administrators (or very close to them anyway) and recognize data's gone missing before they hoped. In larger orgs/projects, you'd have to have pretty long times between the close and the delete for someone to notice, figure out why the data was gone or if it was a security configuration or someone changing index patterns in Kibana, and then doing the right thing to stop deletion (e.g. not just reopening the index, but also disabling the policy for the index). I suspect in most cases where there are larger projects, the cluster administrator would just want to ensure that a snapshot had occurred on the index before deletion.

[dakrone]

Closing this as we don't think there's enough interest in this.