Bugfix: Pass docvalue_fields for elasticsearch #404
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
We had a bug reported in #388 (comment) where logs queries to ElasticSearch v7.8.1 with OpenSearch plugin were returning No Data. The reason for this was that we're sending "fields" here to OpenSearch with the timestamp format we desire.
ElasticSearch api, however, expects
docvalue_fields
instead, so it returns no data. Older versions of Elastic expectfields
as well.There was also some refactoring, since without it, the time field was in some cases added twice in
fields
ordocvalue_fields
(e.g.docvalue_fields: ["@timestamp", {"field": "@timestamp", "format": "date_time"}]
timeField
toAddTimeFieldWithStandardizedFormat
, where the field, along with its formatting will be added according to the flavor and version. Still not sure iffielddata_fields
is completely necessary, but since it's only for Elastic < 5 and logs queries, it might not be relevant in time if we decide to deprecate. The same goes for the new version ofAddDocValueFields
functionscript_fields
field since it doesn't seem to be used anywhereWhich issue(s) this PR fixes:
Fixes #405
Special notes for your reviewer:
To test with Elastic 7.8.1: