Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix back quoted alias of FROM subquery #1189

Merged

Conversation

dai-chen
Copy link
Collaborator

@dai-chen dai-chen commented Dec 16, 2022

Description

Previously, FROM subquery alias was not unquoted in AST layer. This caused back quoted symbol name pushed to symbol table and thus fall back to legacy subquery logic due to semantic exception.

Example

Take SELECT `a`.`name` FROM (...) AS `a` for example. In AST node, alias name of RelationSubquery is still `a` and then Analyzer push `a` to symbol table. When resolving a in a.name, semantic exception is thrown because there is `a` instead of a in symbol table. Please check the comments in #550 below for more details.

Testing

The comparison test seems broken due to failed OpenSearch address resolving. I made local changes and ran it manually. Need to send separate PR to fix the comparison test framework.

$ ./gradlew :integ-test:comparisonTest -Dqueries=bugfixes/550.txt

...
{
  "summary": {
    "total": 2,
    "failure": 0,
    "success": 2
  },
  "tests": [
    {
      "result": "Success",
      "id": 1,
      "sql": "SELECT ABS(`flights`.`AvgTicketPrice`) FROM (SELECT `AvgTicketPrice` FROM `opensearch_dashboards_sample_data_flights`) AS `flights` GROUP BY ABS(`flights`.`AvgTicketPrice`)"
    },
    {
      "result": "Success",
      "id": 2,
      "sql": "SELECT `b`.`Origin`, `b`.`avgPrice` FROM (SELECT `a`.`Origin` AS `Origin`, AVG(`AvgTicketPrice`) AS `avgPrice` FROM (SELECT `Origin`, `AvgTicketPrice` FROM `opensearch_dashboards_sample_data_flights` WHERE `FlightDelay` = True) AS `a` GROUP BY `a`.`Origin`) AS `b` ORDER BY `b`.`avgPrice` DESC"
    }
  ]
}

Documentation

I don't see the limitation mentioned in our doc any more. Even very complex and nested subquery as below can be supported now. So I updated the doc to remove the limitation statement.

POST _plugins/_sql
{
  "query": """
    SELECT b.Origin, b.avgPrice
    FROM (
      SELECT
        a.Origin AS Origin,
        AVG(AvgTicketPrice) AS avgPrice
      FROM (
        SELECT Origin, AvgTicketPrice
        FROM opensearch_dashboards_sample_data_flights
        WHERE FlightDelay = True
      ) AS a
      GROUP BY a.Origin
    ) AS b
    ORDER BY b.avgPrice DESC
  """
}

{
  "schema": [
    {
      "name": "Origin",
      "type": "keyword"
    },
    {
      "name": "avgPrice",
      "type": "double"
    }
  ],
  "datarows": [
    [
      "Charles de Gaulle International Airport",
      999.1396484375
    ],
    [
      "Huntsville International Carl T Jones Field",
      993.1741333007812
    ],
    [
      "Minneapolis-St Paul International/Wold-Chamberlain Airport",
      952.474853515625
    ],
...

Explain output:
{
  "root": {
    "name": "ProjectOperator",
    "description": {
      "fields": "[Origin, avgPrice]"
    },
    "children": [
      {
        "name": "SortOperator",
        "description": {
          "sortList": {
            "avgPrice": {
              "sortOrder": "DESC",
              "nullOrder": "NULL_LAST"
            }
          }
        },
        "children": [
          {
            "name": "ProjectOperator",
            "description": {
              "fields": "[Origin, avgPrice]"
            },
            "children": [
              {
                "name": "AggregationOperator",
                "description": {
                  "aggregators": "[AVG(AvgTicketPrice)]",
                  "groupBy": "[Origin]"
                },
                "children": [
                  {
                    "name": "ProjectOperator",
                    "description": {
                      "fields": "[Origin, AvgTicketPrice]"
                    },
                    "children": [
                      {
                        "name": "OpenSearchIndexScan",
                        "description": {
                          "request": """OpenSearchQueryRequest(indexName=opensearch_dashboards_sample_data_flights, sourceBuilder={"from":0,"size":200,"timeout":"1m","query":{"term":{"FlightDelay":{"value":true,"boost":1.0}}},"_source":{"includes":["Origin","AvgTicketPrice"],"excludes":[]},"sort":[{"_doc":{"order":"asc"}}]}, searchDone=false)"""
                        },
                        "children": []
                      }
                    ]
                  }
                ]
              }
            ]
          }
        ]
      }
    ]
  }
}

Issues Resolved

Check List

  • New functionality includes testing.
    • All tests pass, including unit test, integration test and doctest
  • New functionality has been documented.
    • New functionality has javadoc added
    • New functionality has user manual doc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Chen Dai <daichen@amazon.com>
Signed-off-by: Chen Dai <daichen@amazon.com>
@dai-chen dai-chen added the bug Something isn't working label Dec 16, 2022
@dai-chen dai-chen self-assigned this Dec 16, 2022
@codecov-commenter
Copy link

codecov-commenter commented Dec 16, 2022

Codecov Report

Merging #1189 (1d36d7a) into main (034027c) will decrease coverage by 0.00%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##               main    #1189      +/-   ##
============================================
- Coverage     95.81%   95.81%   -0.01%     
  Complexity     3521     3521              
============================================
  Files           352      352              
  Lines          9359     9358       -1     
  Branches        673      673              
============================================
- Hits           8967     8966       -1     
  Misses          334      334              
  Partials         58       58              
Flag Coverage Δ
query-workbench 62.76% <ø> (ø)
sql-engine 98.31% <100.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...java/org/opensearch/sql/sql/parser/AstBuilder.java 100.00% <100.00%> (ø)
...pensearch/sql/ppl/parser/AstExpressionBuilder.java 100.00% <0.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Signed-off-by: Chen Dai <daichen@amazon.com>
@dai-chen dai-chen marked this pull request as ready for review December 16, 2022 21:10
@dai-chen dai-chen requested a review from a team as a code owner December 16, 2022 21:10
Signed-off-by: Chen Dai <daichen@amazon.com>
@dai-chen dai-chen merged commit 91ef34d into opensearch-project:main Dec 23, 2022
@dai-chen dai-chen deleted the fix-subquery-alias-backquote branch December 23, 2022 18:01
opensearch-trigger-bot bot pushed a commit that referenced this pull request Dec 23, 2022
* Unquote from subquery alias

Signed-off-by: Chen Dai <daichen@amazon.com>

* Add comparison test case

Signed-off-by: Chen Dai <daichen@amazon.com>

* Add more comparison test case

Signed-off-by: Chen Dai <daichen@amazon.com>

* Update doc to remove limitations

Signed-off-by: Chen Dai <daichen@amazon.com>

Signed-off-by: Chen Dai <daichen@amazon.com>
(cherry picked from commit 91ef34d)
dai-chen added a commit that referenced this pull request Jan 3, 2023
* Unquote from subquery alias

Signed-off-by: Chen Dai <daichen@amazon.com>

* Add comparison test case

Signed-off-by: Chen Dai <daichen@amazon.com>

* Add more comparison test case

Signed-off-by: Chen Dai <daichen@amazon.com>

* Update doc to remove limitations

Signed-off-by: Chen Dai <daichen@amazon.com>

Signed-off-by: Chen Dai <daichen@amazon.com>
(cherry picked from commit 91ef34d)

Co-authored-by: Chen Dai <daichen@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x bug Something isn't working SQL
Projects
None yet
5 participants