Implement write ability for DLS/FLS via an ingest pipeline and special index privileges

[elasticmachine]

Original comment by @geekpete:

Context

First, for context, this is the original source of this idea (Thanks @tvernum !):
LINK REDACTED

I was asking on behalf of a customer if we currently support or will support in the future, write access via DLS/FLS security. The customer was building a comment system against a parent child schema where child docs are comments/tags off read only parent docs. Users would be restricted by DLS to what parent docs they can see and comment on and each user would also only be allowed to edit their own created comments.

Feature

Provide some security related ingest processors for an ingest pipeline that would allow users to perform writes to indices that have special security permissions set to allow a method for users with DLS/FLS restrictions to perform controlled writes to indices.

So a privilege on an index that said “allow write, but only via pipeline XYZ” and then allowed the cluster admin to write a script processor that references the user’s roles, they’d get something loosely like an insert/update trigger in SQL.

[tvernum]

I took at look at how feasible this would be, and it runs into a problem with the way we handle bulk request (and, by consequence all DocWriteRequests as these get internally converted into Bulk)

If we assume as a given that we'd want to support this for bulk (and we should) then we have 2 places where the AuthorizationService could attempt to handle it

1. On first processing of the BulkRequest
2. When processing the BulkShardRequest

The former is complicated by the fact that a BulkRequest has no way of marking an item as "failed". So if the AuthorizationService detected that an item within the bulk was invalid, it would need to fail the whole bulk request, which is undesirable.

The latter is complicated by the fact that the PipelineExecutionService clears the pipeline from the bulk item after processing it, so the AuthorizationService cannot tell whether at item went through a pipeline or not.

Neither of those are insurmountable, but during the implementation of #26434, the general view was that the AuthorizationService should behave like an interceptor, which would allow it to "modify" requests in a similar way to TransportBulkAction.BulkRequestModifier.
To make progress on this issue, we would first need to do work to allow authorization decisions to take place in the context of an interceptor.

[tvernum]

Pinging @elastic/es-security (because @elasticmachine has failed to do so)

[pcsanwald]

We discussed this in FixIt Friday, and the consensus is, for now, the difficulty in implementing this outweighs the benefits it would provide.