Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add tests for the supported query types. #35319

Merged
merged 5 commits into from
Nov 8, 2018

Conversation

jtibshirani
Copy link
Contributor

The following query types should now be covered:

  • term, terms and terms_set
  • prefix
  • range
  • match and multi_match
  • query_string and simple_query_string
  • exists

I preferred unit tests as much as possible, but certain query types (like match and query_string) were more natural to check through integration tests.

@jtibshirani jtibshirani added >test Issues or PRs that are addressing/adding tests :Search Foundations/Mapping Index mappings, including merging and defining field types labels Nov 6, 2018
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search-aggs

boolean includeLower,
boolean includeUpper,
QueryShardContext context) {
if (lowerTerm == null || upperTerm == null) {
Copy link
Contributor Author

@jtibshirani jtibshirani Nov 6, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could fix this instead of disallowing unbounded range queries, but I wasn't sure the extra complexity was worth it. It's not clear how useful this range functionality will be right now, especially without support for numerics.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like this limitation is ok for now. We can revisit this later if necessary

@jtibshirani jtibshirani force-pushed the json-field-tests branch 4 times, most recently from 909c276 to f693e46 Compare November 6, 2018 23:23
Copy link
Contributor

@colings86 colings86 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left a comment but otherwise I think this is good. I'd like @romseygeek to take a look too though

boolean includeLower,
boolean includeUpper,
QueryShardContext context) {
if (lowerTerm == null || upperTerm == null) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like this limitation is ok for now. We can revisit this later if necessary

TermRangeQuery expected = new TermRangeQuery("field",
new BytesRef("key\0lower"),
new BytesRef("key\0upper"), false, false);
assertEquals(expected, ft.rangeQuery("lower", "upper", false, false, null));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we also check that we do the right thing if the bounds are inclusive?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

.startObject()
.startObject("headers")
.field("content-type", "application/json")
.endObject()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not especially related to the current PR, but how is case normalization being handled? The normal HTTP header here is Content-type with a capital C; is there a lowercase filter being applied anywhere, or is that being ignored for the moment?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't apply any lowercasing to the object keys. This is consistent with the way normal field names are handled in the mapping, which are not lowercased (you could have two distinct fields for example, one named Content-Type and the other content-type). Do you think an option to automatically case-fold the keys would be useful?

Separately for field values (not keys), keyword fields provide a normalizer option. This is something we could consider, although it might need to be designed differently as json fields can contain non-string values.

Copy link
Contributor

@romseygeek romseygeek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I have one completely unrelated comment to do with case normalization, but feel free to answer that elsewhere...

@jtibshirani
Copy link
Contributor Author

Thank you both for reviewing, I plan to fix the failing test and merge.

This affects some of the checks in tests, but doesn't have an actual effect
on indexing (because there we rely on FieldType#docValuesType).
@jtibshirani jtibshirani merged commit e7613b7 into elastic:object-fields Nov 8, 2018
@jtibshirani jtibshirani deleted the json-field-tests branch November 8, 2018 00:42
jtibshirani added a commit to jtibshirani/elasticsearch that referenced this pull request Nov 8, 2018
* Add tests for the supported query types.
* Disallow unbounded range queries on keyed JSON fields.
* Make sure MappedFieldType#hasDocValues always returns false.
jtibshirani added a commit to jtibshirani/elasticsearch that referenced this pull request Nov 9, 2018
* Add tests for the supported query types.
* Disallow unbounded range queries on keyed JSON fields.
* Make sure MappedFieldType#hasDocValues always returns false.
jtibshirani added a commit that referenced this pull request Nov 30, 2018
* Add tests for the supported query types.
* Disallow unbounded range queries on keyed JSON fields.
* Make sure MappedFieldType#hasDocValues always returns false.
jtibshirani added a commit that referenced this pull request Dec 5, 2018
* Add tests for the supported query types.
* Disallow unbounded range queries on keyed JSON fields.
* Make sure MappedFieldType#hasDocValues always returns false.
jtibshirani added a commit that referenced this pull request Dec 13, 2018
* Add tests for the supported query types.
* Disallow unbounded range queries on keyed JSON fields.
* Make sure MappedFieldType#hasDocValues always returns false.
jtibshirani added a commit that referenced this pull request Mar 6, 2019
* Add tests for the supported query types.
* Disallow unbounded range queries on keyed JSON fields.
* Make sure MappedFieldType#hasDocValues always returns false.
jtibshirani added a commit that referenced this pull request Mar 7, 2019
* Add tests for the supported query types.
* Disallow unbounded range queries on keyed JSON fields.
* Make sure MappedFieldType#hasDocValues always returns false.
jtibshirani added a commit that referenced this pull request Mar 9, 2019
* Add tests for the supported query types.
* Disallow unbounded range queries on keyed JSON fields.
* Make sure MappedFieldType#hasDocValues always returns false.
jtibshirani added a commit that referenced this pull request Mar 11, 2019
* Add tests for the supported query types.
* Disallow unbounded range queries on keyed JSON fields.
* Make sure MappedFieldType#hasDocValues always returns false.
jtibshirani added a commit that referenced this pull request Mar 14, 2019
* Add tests for the supported query types.
* Disallow unbounded range queries on keyed JSON fields.
* Make sure MappedFieldType#hasDocValues always returns false.
jtibshirani added a commit that referenced this pull request Mar 19, 2019
* Add tests for the supported query types.
* Disallow unbounded range queries on keyed JSON fields.
* Make sure MappedFieldType#hasDocValues always returns false.
jtibshirani added a commit to jtibshirani/elasticsearch that referenced this pull request Mar 19, 2019
* Add tests for the supported query types.
* Disallow unbounded range queries on keyed JSON fields.
* Make sure MappedFieldType#hasDocValues always returns false.
jtibshirani added a commit that referenced this pull request Mar 28, 2019
* Add tests for the supported query types.
* Disallow unbounded range queries on keyed JSON fields.
* Make sure MappedFieldType#hasDocValues always returns false.
jtibshirani added a commit that referenced this pull request Apr 10, 2019
* Add tests for the supported query types.
* Disallow unbounded range queries on keyed JSON fields.
* Make sure MappedFieldType#hasDocValues always returns false.
jtibshirani added a commit that referenced this pull request Apr 17, 2019
* Add tests for the supported query types.
* Disallow unbounded range queries on keyed JSON fields.
* Make sure MappedFieldType#hasDocValues always returns false.
jtibshirani added a commit that referenced this pull request May 1, 2019
* Add tests for the supported query types.
* Disallow unbounded range queries on keyed JSON fields.
* Make sure MappedFieldType#hasDocValues always returns false.
jtibshirani added a commit that referenced this pull request May 24, 2019
* Add tests for the supported query types.
* Disallow unbounded range queries on keyed JSON fields.
* Make sure MappedFieldType#hasDocValues always returns false.
jtibshirani added a commit that referenced this pull request May 29, 2019
* Add tests for the supported query types.
* Disallow unbounded range queries on keyed JSON fields.
* Make sure MappedFieldType#hasDocValues always returns false.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Search Foundations/Mapping Index mappings, including merging and defining field types >test Issues or PRs that are addressing/adding tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants