include_in_root with geo_shape is broken in 7

[pmishev]

Elasticsearch version : 7.11.1

OS version: Ubuntu

Looks like include_in_root can no longer be used in conjunction with geo_shape fields, as removing any of those doesn't produce an error.

Reproducing the problem:

PUT ttt
{
  "mappings": {
    "properties": {
      "addresses": {
        "type": "nested",
        "include_in_root": true,
        "properties": {
          "name": {
            "type": "text"
          },
          "location": {
            "type": "geo_shape"
          }
        }
      }
    }
  }
}


PUT ttt/_doc/1
{
  "addresses": [
    {
      "name": "aaa",
      "location": {
        "type": "point",
        "coordinates": [
          150.121212,
          -30.121212
        ]
      }
    },
    {
      "name": "bbb",
      "location": {
        "type": "point",
        "coordinates": [
          150.121212,
          -30.121212
        ]
      }
    }
  ]
}

Results in:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "illegal_argument_exception",
        "reason" : "DocValuesField \"addresses.location\" appears more than once in this document (only one value is allowed per field)"
      }
    ],
    "type" : "illegal_argument_exception",
    "reason" : "DocValuesField \"addresses.location\" appears more than once in this document (only one value is allowed per field)"
  },
  "status" : 400
}

The equivalent for ES 6.8, which works as expected with no errors:

PUT ttt
{
  "mappings": {
    "_doc": {
      "properties": {
        "addresses": {
          "type": "nested",
          "include_in_root": true,
          "properties": {
            "name": {
              "type": "text"
            },
            "location": {
              "type": "geo_shape"
            }
          }
        }
      }
    }
  }
}


PUT ttt/_doc/1
{
  "addresses": [
    {
      "name": "aaa",
      "location": {
        "type": "point",
        "coordinates": [
          150.121212,
          -30.121212
        ]
      }
    },
    {
      "name": "bbb",
      "location": {
        "type": "point",
        "coordinates": [
          150.121212,
          -30.121212
        ]
      }
    }
  ]
}

[pmishev]

I believe I have further narrowed down the issue to the new way geo_shapes are indexed (https://www.elastic.co/guide/en/elasticsearch/reference/current/breaking-changes-7.0.html#changed-default-geo-shape-index-strategy).
When I specify the tree option to a deprecated type, the error is no longer seen:

          "location": {
            "type": "geo_shape",
            "tree": "geohash"
          },

[elasticmachine]

Pinging @elastic/es-analytics-geo (Team:Analytics)

[iverase]

The issue is that in the new way geo_shapes are indexed, doc values are generated by default and in the combination above, they are included twice on the root document which leads to the error you are seeing. This is indeed a bug, one way to avoid it, is to explicitly disable doc values:

PUT ttt
{
  "mappings": {
    "properties": {
      "addresses": {
        "type": "nested",
        "include_in_root": true,
        "properties": {
          "name": {
            "type": "text"
          },
          "location": {
            "type": "geo_shape",
            "doc_values" : false
          }
        }
      }
    }
  }
}

[elasticmachine]

Pinging @elastic/es-search (Team:Search)

[wchaparro]

@javanna we think this is a mapping problem. Could you take a look pls?

[javanna]

Closing as a duplicate of #89164 . The problem still reproduces in more recent versions.