ESUTF8StreamJsonParser does not reset correctly the last optimised value.

[gmarouli]

Elasticsearch Version

main, 9.1, 8.19

Installed Plugins

No response

Java Version

bundled

OS Version

n/a

Problem Description

The last optimised value does not get properly reset and its reused later in different fields.

Steps to Reproduce

# Create an index with the following mapping:
PUT /my-index
{
  "mappings": {
    "_data_stream_timestamp": {
      "enabled": true
    },
    "properties": {
      "@timestamp": {
        "type": "date"
      },
      "my-funky-field": {
        "type": "keyword",
        "fields": {
          "subfield_text": {
            "type": "text"
          }
        }
      },
      "my-poor-field": {
        "type": "ip"
      }
    }
  }
}

# Index the following document
POST /my-index/_doc
{
  "@timestamp": "2024-01-01T10:00:52.000Z",
  "my-funky-field": "14`\\%+",
  "my-poor-field": [
      "184.123.133.116"
    ]
}

Then we see the error:

"caused_by": {
      "type": "illegal_argument_exception",
      "reason": "'14`\\%+' is not an IP string literal."
    }

Logs (if relevant)

No response

[elasticsearchmachine]

Pinging @elastic/es-storage-engine (Team:StorageEngine)

[elasticsearchmachine]

Pinging @elastic/es-core-infra (Team:Core/Infra)

[gmarouli]

This bug is occurring because we are calling com.fasterxml.jackson.core.json.UTF8StreamJsonParser#getText() which resets the _tokenIncomplete outside of the scope of ESUTF8StreamJsonParser. This means that if we call getText() write after an optimised value, this optimised value will not be reset.

    @Override
    public String getText() throws IOException
    {
        if (_currToken == JsonToken.VALUE_STRING) {
            if (_tokenIncomplete) {
                _tokenIncomplete = false;
                return _finishAndReturnString(); // only strings can be incomplete
            }
            return _textBuffer.contentsAsString();
        }
        return _getText2(_currToken);
    }