Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add raw_document query support to backend #214

Merged
merged 12 commits into from
Jul 17, 2023
Merged

Conversation

fridgepoet
Copy link
Member

@fridgepoet fridgepoet commented Jul 13, 2023

What this PR does / why we need it:
This recreates the raw_document query processing that exists in the frontend in the backend. It redirects any queries with only raw_document to the backend. It's very similar to the recent raw_data work.

The query building is based on its equivalent in Elasticsearch and the original PR is here.
The result processing is based on its equivalent in Elasticsearch. The original PR is here.

Which issue(s) this PR fixes:

Fixes #195

Special notes for your reviewer:

  • As mentioned in the Elasticsearch PR, the order of the keys in the JSON will not be the same between the previous frontend implementation and this backend PR.

    results from backend are alphabetically ordered - this is done automatically while JSON.Marshall. From docs: ...The map keys are sorted. This shouldn't be an issue at all, as the order of fields in json is not used anywhere.

  • Another difference is that the timestamp will be an array, not just the value, same as in the Elasticsearch PR.

    In json field (only field), @timestamp is array with value when processed trough backend (compared to just value when processed trough frontend).

    Although we can match the same behavior as the frontend by supplying only the single value from the array, if this more direct result has been acceptable for Elasticsearch users, the code will be simpler.

@fridgepoet fridgepoet changed the title wip raw_doc query Add raw_document query support to backend Jul 13, 2023
Base automatically changed from sort-array to main July 14, 2023 15:04
@fridgepoet fridgepoet marked this pull request as ready for review July 14, 2023 15:07
@fridgepoet fridgepoet requested a review from a team as a code owner July 14, 2023 15:07
@fridgepoet fridgepoet requested review from iwysiu, kevinwcyu and idastambuk and removed request for a team July 14, 2023 15:07
Copy link
Contributor

@iwysiu iwysiu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly just style nits again, with one question about the hit fields

pkg/opensearch/lucene_handler.go Outdated Show resolved Hide resolved
pkg/opensearch/response_parser.go Outdated Show resolved Hide resolved
pkg/opensearch/response_parser.go Show resolved Hide resolved
pkg/opensearch/response_parser.go Outdated Show resolved Hide resolved
@fridgepoet fridgepoet merged commit cf5462c into main Jul 17, 2023
@fridgepoet fridgepoet deleted the raw_document_backend branch July 17, 2023 16:12
@fridgepoet fridgepoet mentioned this pull request Jul 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Backend: Implement Lucene raw_document query
2 participants