[BUG] "Field level security" and "Field masking definitions" don't work together with "Document level security"

[rafaelma]

What is the bug?
"Field level security (FLS)" and "Field masking definitions(FMD)" don't work as expected, together with "Document level security(DLS)"

FLS/FMD get applied only to the index-pattern definition in the role and bypass the DLS definition

When having 2 roles using "Document level security" to restrict the role to a subset of documents in an index, and one of them also uses "Field level security" and/or "Field masking definitions", the restriction to the fields defined in one of the roles will affect the other role as well.

How can one reproduce the bug?
Steps to reproduce the behavior:

1. Create an index "logs-component1"
2. Update the index with 2 documents having these two fields:

{
    "product": "product-1",
    "description": "This is a document for product-1"
}

{
    "product": "product-2",
    "description": "This is a document for product-2"
}

3. Create 2 roles, A and B:

{
  "A": {
    "reserved": false,
    "hidden": false,
    "cluster_permissions": [],
    "index_permissions": [
      {
        "index_patterns": [
          "logs-component1"
        ],
        "dls": """{
           "bool": {
             "must": [
               {
                 "match": {
                 "product": "product-1"
                 }
               }
             ]
            }
           }""",
        "fls": [],
        "masked_fields": [],
        "allowed_actions": [
          "data_access"
        ]
      }
    ],
    "tenant_permissions": [],
    "static": false
  }
}

{
  "B": {
    "reserved": false,
    "hidden": false,
    "cluster_permissions": [],
    "index_permissions": [
      {
        "index_patterns": [
          "logs-component1"
        ],
        "dls": """{
           "bool": {
             "must": [
               {
                 "match": {
                 "product": "product-2"
                 }
               }
             ]
            }
           }""",
        "fls": [],
        "masked_fields": [
          "description"
        ],
        "allowed_actions": [
          "data_access"
        ]
      }
    ],
    "tenant_permissions": [],
    "static": false
  }
}

4. Mapp user "rafael" to role A and B:

{
  "A": {
    "hosts": [],
    "users": [
      "rafael"
    ],
    "reserved": false,
    "hidden": false,
    "backend_roles": [],
    "and_backend_roles": []
  }
}

{
  "B": {
    "hosts": [],
    "users": [
      "rafael"
    ],
    "reserved": false,
    "hidden": false,
    "backend_roles": [],
    "and_backend_roles": []
  }
}

5. Search all documents in the index

{
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 2,
      "relation": "eq"
    },
    "max_score": 2,
    "hits": [
      {
        "_index": "logs-component1",
        "_id": "cYWcRYoBA8cFetXWaUrZ",
        "_score": 2,
        "_source": {
          "product": "product-1",
          "description": "fa845b9b2b564e179180c26858748e79ce271d1cd3d3efaf72ea2dd031c18fda"
        }
      },
      {
        "_index": "logs-component1",
        "_id": "coWcRYoBA8cFetXWe0o6",
        "_score": 2,
        "_source": {
          "product": "product-2",
          "description": "fa845b9b2b564e179180c26858748e79ce271d1cd3d3efaf72ea2dd031c18fda"
        }
      }
    ]
  }
}

The document with "_id": "cYWcRYoBA8cFetXWaUrZ" should not get "description" masked because the user gets access to this document via role-A.

What is the expected behavior?

Use case:

• We have an index with information about a component used by different products. The products are identified by a field ("product").
• We have two roles A and B.
• A will give full access to all documents in the index with "product"="product-1"
• B will give access to all documents in an index with "product"="product-2" but will restrict the access to one field in the index via "Field level security" or "Field anonymization"
• An user gets mapped to role A and B
• The user should get full access to all documents in the index with "product"="product-1", and restricted access to all documents with "product"="product-2"

What is your host/environment?

• OS: Ubuntu 22.04.2 LTS
• Opensearch Version: 2.9
• Plugins:

$ /usr/share/opensearch/bin/opensearch-plugin list -v |egrep "Name|^Version"

Name: opensearch-alerting
Version: 2.9.0.0
Name: opensearch-anomaly-detection
Version: 2.9.0.0
Name: opensearch-asynchronous-search
Version: 2.9.0.0
Name: opensearch-cross-cluster-replication
Version: 2.9.0.0
Name: opensearch-geospatial
Version: 2.9.0.0
Name: opensearch-index-management
Version: 2.9.0.0
Name: opensearch-job-scheduler
Version: 2.9.0.0
Name: opensearch-knn
Version: 2.9.0.0
Name: opensearch-ml
Version: 2.9.0.0
Name: opensearch-neural-search
Version: 2.9.0.0
Name: opensearch-notifications
Version: 2.9.0.0
Name: opensearch-notifications-core
Version: 2.9.0.0
Name: opensearch-observability
Version: 2.9.0.0
Name: opensearch-performance-analyzer
Version: 2.9.0.0
Name: opensearch-reports-scheduler
Version: 2.9.0.0
Name: opensearch-security
Version: 2.9.0.0
Name: opensearch-security-analytics
Version: 2.9.0.0
Name: opensearch-sql
Version: 2.9.0.0

Do you have any screenshots?
If applicable, add screenshots to help explain your problem.

Do you have any additional context?
Add any other context about the problem.

[peternied]

How two different rule sets are combined at runtime based on role A and role B make it hard for there to be an obvious answer to 'how are the rules applied'. I think that unless there is a singular place where these rules are described admins and users alike will have trouble handling these scenarios - Views [1] might be a way to address this.

@rafaelma What do you think?

• Data projection with views OpenSearch#6181

[rafaelma]

@peternied Thanks for your feedback :-) . The view functionality you describe in opensearch-project/OpenSearch#6181 will move the document level security (DLS), field level security (FLS), and field masking definitions (FMD) out of a "Role definition" and into a "View definition".

It is a cleaner separation compare to what is available today and different roles could get access to the same DLS/FLS/FMD definition and avoid the need to define the same DLS/FLS/FMD in different roles. It is obviously an improving in the access management functionality.

But this functionality does not help with the bug described in this issue. The FLS/FMD gets applied only to the index-pattern definition in the role and bypass the DLS definition, we think this is obviously a bug.

[stephen-crawford]

[Triage] Thank you for filing this issue @rafaelma. This issue appears to be a result of the role combinations Peter mentioned above. There is not a clear path forward to fixing the non-deterministic behavior that we currently have since the ordering of the role impacts the results. We can address this moving forward by marking as help wanted but leaving untriaged since it is certainly something worth improving but the action criteria for this is not totally clear at the moment.