-
Notifications
You must be signed in to change notification settings - Fork 429
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
use JSON field for concat field #4924
Labels
Comments
PSeitz
added a commit
that referenced
this issue
May 6, 2024
use JSON for concat fields instead of concatenating text fields. With quickwit-oss/tantivy#2383 we support now non-object values on the root in the JSON field. As a nice side-effect, this will make regular JSON fields more powerful. Instead only for nested types, JSON is now also useable for flat mixed-type fields. Closes: #4924
PSeitz
added a commit
that referenced
this issue
May 6, 2024
use JSON for concat fields instead of concatenating text fields. With quickwit-oss/tantivy#2383 we support now non-object values on the root in the JSON field. As a nice side-effect, this will make regular JSON fields more powerful. Instead only for nested types, JSON is now also useable for flat mixed-type fields. Closes: #4924
PSeitz
added a commit
that referenced
this issue
May 6, 2024
use JSON for concat fields instead of concatenating text fields. With quickwit-oss/tantivy#2383 we support now non-object values on the root in the JSON field. As a nice side-effect, this will make regular JSON fields more powerful. Instead only for nested types, JSON is now also useable for flat mixed-type fields. Closes: #4924
PSeitz
added a commit
that referenced
this issue
May 6, 2024
use JSON for concat fields instead of concatenating text fields. With quickwit-oss/tantivy#2383 we support now non-object values on the root in the JSON field. As a nice side-effect, this will make regular JSON fields more powerful. Instead only for nested types, JSON is now also useable for flat mixed-type fields. Closes: #4924
PSeitz
added a commit
that referenced
this issue
May 13, 2024
use JSON for concat fields instead of concatenating text fields. With quickwit-oss/tantivy#2383 we support now non-object values on the root in the JSON field. As a nice side-effect, this will make regular JSON fields more powerful. Instead only for nested types, JSON is now also useable for flat mixed-type fields. Closes: #4924
PSeitz
added a commit
that referenced
this issue
Jun 8, 2024
use JSON for concat fields instead of concatenating text fields. With quickwit-oss/tantivy#2383 we support now non-object values on the root in the JSON field. As a nice side-effect, this will make regular JSON fields more powerful. Instead only for nested types, JSON is now also useable for flat mixed-type fields. Closes: #4924
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The concat fields introduced in this PR #4773 uses a multi-valued text field to concatenate the data into one field. Non-string types are converted and indexed as string type.
We could use the JSON field instead of a multivalued text field for the concat field.
That way we keep the type info and there are some synergies since JSON has the same problem space when indexing and querying data with mixed types.
e.g. how would we know if the user input in a query is a text, number, date, etc. and how to generate queries that can handle different types with potential ambiguous cases.
On the indexing side it is similar, that we may encounter e.g. "2024-04-28T17:31:59.893Z" and should convert that to a date.
The change should be straightforward: Change the field type to JSON and remove the conversion logic.
Impact
Impact should be minimal, since the feature is pretty new and not yet released.
But may be some customers are already using it. @trinity-1686a @fmassot
The field type would change, but existing usage should still work, since we check the fields type anyways?
Update: There is one user using it, and it's okay to break it (which I think doesn't even happen).
The text was updated successfully, but these errors were encountered: