Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SQL: normalized keywords shouldn't be allowed for groupings and sorting #35203

Closed
astefan opened this issue Nov 2, 2018 · 1 comment
Closed
Labels

Comments

@astefan
Copy link
Contributor

astefan commented Nov 2, 2018

A normalized keyword field can potentially change the original value of a field, but SQL works with exact values. Probably as part of #34718, the restriction was lifted, as field_caps API can tell if a field is aggregatable/searchable, but not if it's normalized or a simple keyword.

As such, in the following scenario:

{
  "settings": {
    "analysis": {
      "normalizer": {
        "my_normalizer": {
          "type": "custom",
          "filter": [
            "lowercase",
            "asciifolding"
          ]
        }
      }
    }
  },
  "mappings": {
    "test": {
      "properties": {
        "user": {
          "type": "text",
          "fields": {
            "normalized": {
              "type": "keyword",
              "normalizer": "my_normalizer"
            },
            "keyword": {
              "type": "keyword"
            }
          }
        },
        "user2": {
          "type": "keyword",
          "normalizer": "my_normalizer"
        }
      }
    }
  }
}

With a query like { "query" : "select user from test group by user" } one gets back:

{
    "error": {
        "root_cause": [
            {
                "type": "mapping_exception",
                "reason": "Multiple exact keyword candidates available for [user]; specify which one to use"
            }
        ],
        "type": "mapping_exception",
        "reason": "Multiple exact keyword candidates available for [user]; specify which one to use"
    },
    "status": 400
}

And for { "query" : "select user2 from test group by user2" } there are results returned where, in fact, ES SQL should return an error.

@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search-aggs

@lcawl lcawl added v6.5.2 and removed v6.5.1 labels Nov 20, 2018
@tomcallahan tomcallahan added v6.5.3 and removed v6.5.2 labels Dec 4, 2018
@jasontedor jasontedor added v8.0.0 and removed v7.0.0 labels Feb 6, 2019
@matriv matriv closed this as completed in 10ab391 Feb 28, 2019
matriv added a commit that referenced this issue Mar 1, 2019
For functions: move checks for `text` fields without underlying `keyword`
fields or with many of them (ambiguity) to the type resolution stage.

For Order By/Group By: move checks to the `Verifier` to catch early
before `QueryTranslator` or execution.

Closes: #38501
Fixes: #35203
matriv added a commit that referenced this issue Mar 1, 2019
For functions: move checks for `text` fields without underlying `keyword`
fields or with many of them (ambiguity) to the type resolution stage.

For Order By/Group By: move checks to the `Verifier` to catch early
before `QueryTranslator` or execution.

Closes: #38501
Fixes: #35203
matriv added a commit that referenced this issue Mar 1, 2019
For functions: move checks for `text` fields without underlying `keyword`
fields or with many of them (ambiguity) to the type resolution stage.

For Order By/Group By: move checks to the `Verifier` to catch early
before `QueryTranslator` or execution.

Closes: #38501
Fixes: #35203
matriv added a commit that referenced this issue Mar 1, 2019
For functions: move checks for `text` fields without underlying `keyword`
fields or with many of them (ambiguity) to the type resolution stage.

For Order By/Group By: move checks to the `Verifier` to catch early
before `QueryTranslator` or execution.

Closes: #38501
Fixes: #35203
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

7 participants