Two single or double quote escapes single or double quote when string is surrounded by same type of quote #696

MitchellGale · 2022-07-15T20:11:50Z

Description

Adds implementation for double quotes working to escape a single quote.

Uses string builder to generate query in String Utils. Only escapes quote when outside quotes are matching, does not work for `. Below are examples of how the new queries function.

SELECT 'I''m';          -> I'm
SELECT 'I''''m';        -> I''m
SELECT "I''m";          -> I''m
SELECT "I""m";          -> I"m
SELECT \`I\`\`m\`;      -> I``m
SELECT 'I""''m';        -> I""'m

Issues Resolved

[Issue #682]

Check List

New functionality includes testing.
- All tests pass, including unit test, integration test and doctest
New functionality has been documented.
- New functionality has javadoc added
- New functionality has user manual doc added
Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

joshuali925

lgtm

penghuo · 2022-07-21T20:29:37Z

could you add docs to explain the unquote also?

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Signed-off-by: Andrew Carbonetto <andrewc@AndrewsMBP2022.local>

… is now expected number. 'I\'m' -> I'm 'I\\''m' -> I\'m 'I\\\'m' -> I\'m Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

…y unquoteText. Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

…der in StringUtilsTest.java Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

…quote. Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

… static and non-static imports Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

… Added case for double, double quote being converted to single, double quote. Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

…gle of their respective type when the string is quoted by the same type. Added some unit tests to test for that condition in StringUtilsTest.java. Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

…gine.java. Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

…gle). Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

…l as double quotes. Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

… StringUtils.java. Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

…rrounded by same string. Consistent with MySQL standard. Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

…rom if statement to break off for a return. Added unit test, removed unit tests for whichquote function. Removed whichquote function. Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

…back tick to skip over String Builder. Fixed issue with strings passed to unquotetext being too small casuing array bounds access issue. Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

… double quote escape implementation. Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

codecov-commenter · 2022-07-22T18:27:00Z

Codecov Report

Merging #696 (ba86654) into main (592c58b) will increase coverage by 0.00%.
The diff coverage is n/a.

@@            Coverage Diff            @@
##               main     #696   +/-   ##
=========================================
  Coverage     94.74%   94.75%           
- Complexity     2857     2858    +1     
=========================================
  Files           283      283           
  Lines          7676     7678    +2     
  Branches        560      560           
=========================================
+ Hits           7273     7275    +2     
  Misses          349      349           
  Partials         54       54

Flag	Coverage Δ
query-workbench	`62.76% <ø> (ø)`
sql-engine	`97.74% <ø> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...ilter/lucene/relevance/SimpleQueryStringQuery.java	`100.00% <0.00%> (ø)`

Help us with your feedback. Take ten seconds to tell us how you rate us.

… Integ-DoubleQuote

Yury-Fridlyand · 2022-07-22T23:15:12Z

Please, fix code style:

Error: [checkstyle] [ERROR] /home/runner/work/sql/sql/common/src/main/java/org/opensearch/sql/common/utils/StringUtils.java:30: Line is longer than 100 characters (found 114). [LineLength]
Error: [checkstyle] [ERROR] /home/runner/work/sql/sql/common/src/main/java/org/opensearch/sql/common/utils/StringUtils.java:31: Line is longer than 100 characters (found 109). [LineLength]

…pensearch-project#696 and 317 all together. (opensearch-project#479) (opensearch-project#481) Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com> (cherry picked from commit 646ce3c) Co-authored-by: Yury-Fridlyand <88679692+Yury-Fridlyand@users.noreply.github.com>

…pensearch-project#696 and 317 all together. (opensearch-project#479) Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Disable flaky test in JdbcTestIT. (#475) Signed-off-by: penghuo <penghuo@gmail.com> * Update dependency opensearch-ml-client group name (#477) Signed-off-by: jackieyanghan <jkhanjob@gmail.com> * Collected fixes related to bugs #280, #302, #783, #696 and 317 all together. (#479) Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com> * Add JDK 8 to CI Matrix (#483) (#484) * fixed references in java 8 and added to ci matrix * filter test that fails due to jdk 8 bug * use different degree to radian conversion for difference in jdk * skip date time format tests that cause error due to jdk bug * only disable DateTime unit test on Java 8 Bypassing failed DCO check Signed-off-by: David Cui <davidcui@amazon.com> (cherry picked from commit b6fccbd) Co-authored-by: David Cui <53581635+davidcui1225@users.noreply.github.com> * Add parse docs to PPL commands index (#486) (#487) Signed-off-by: Joshua Li <joshuali925@gmail.com> * Add 1.3.0 release notes (#492) Signed-off-by: Eugene Lee <eugenesk@amazon.com> * Add Kmeans and AD command documentation (#493) (#497) Signed-off-by: jackieyanghan <jkhanjob@gmail.com> (cherry picked from commit ee4bce0) * Bump version to 1.3.1 Signed-off-by: Sayali Gaikawad <gaiksaya@amazon.com> * Bump ml Signed-off-by: Sayali Gaikawad <gaiksaya@amazon.com> * Add new zip Signed-off-by: Sayali Gaikawad <gaiksaya@amazon.com> * Updated release drafter Signed-off-by: Sayali Gaikawad <gaiksaya@amazon.com> * Remove checked-in ml-commons dependency (#529) Signed-off-by: jackieyanghan <jkhanjob@gmail.com> * Bump workbench to 1.3.1 (#540) Signed-off-by: Joshua Li <joshuali925@gmail.com> * Bug Fix, return default ID when log4j ThreadContext is empty (#538) (#539) (#543) Signed-off-by: penghuo <penghuo@gmail.com> (cherry picked from commit 54b7257) * Incremented version to 1.3.2. (#593) Signed-off-by: Zelin Hao <zelinhao@amazon.com> Signed-off-by: Joshua Li <joshuali925@gmail.com> Co-authored-by: Joshua Li <joshuali925@gmail.com> * Version bump 1.3.3 Signed-off-by: pgodithi <pgodithi@amazon.com> * [1.3] Bump spring-beans to 5.2.20 (#632) Signed-off-by: Joshua Li <joshuali925@gmail.com> * Version bump 1.3.3 Signed-off-by: pgodithi <pgodithi@amazon.com> * Incremented version to 1.3.4. (#681) * Incremented version to 1.3.4. Signed-off-by: Zelin Hao <zelinhao@amazon.com> * bump jackson to 2.13.2 Signed-off-by: Joshua Li <joshuali925@gmail.com> Co-authored-by: Joshua Li <joshuali925@gmail.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: Jackie Han <41348518+jackiehanyang@users.noreply.github.com> Co-authored-by: Yury-Fridlyand <88679692+Yury-Fridlyand@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: David Cui <53581635+davidcui1225@users.noreply.github.com> Co-authored-by: Eugene Lee <eugenesk@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: Sayali Gaikawad <61760125+gaiksaya@users.noreply.github.com> Co-authored-by: Joshua Li <joshuali925@gmail.com> Co-authored-by: Zelin Hao <87548827+zelinh@users.noreply.github.com> Co-authored-by: pgodithi <pgodithi@amazon.com> Co-authored-by: Anirudha (Ani) Jadhav <anirudha@nyu.edu>

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

dai-chen · 2022-08-02T21:25:59Z

could you add docs to explain the unquote also?

+1. Probably can add here: https://github.com/opensearch-project/sql/blob/main/docs/user/general/datatypes.rst#string-data-types

… is surrounded by same type of quote (opensearch-project#696) Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>

…pensearch-project#696 and 317 all together. Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

MitchellGale requested a review from a team as a code owner July 15, 2022 20:11

joshuali925 previously approved these changes Jul 21, 2022

View reviewed changes

MitchellGale and others added 27 commits July 22, 2022 08:43

Added integration test for escape character in select statement.

3764343

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

AOS-637: PoC remove escape characters

c952bd4

Signed-off-by: Andrew Carbonetto <andrewc@AndrewsMBP2022.local>

Fixed number of escape characters returned by plugin. Number returned…

8d8c299

… is now expected number. 'I\'m' -> I'm 'I\\''m' -> I\'m 'I\\\'m' -> I\'m Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Addedd test for 'im' in literals.txt.

6fa49f4

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Added unit test for unquoteText in StringUtils.java.

d9871ac

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Removed old unescapeBackslashes from StringUtils which was replaced b…

54c3629

…y unquoteText. Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Removed * from import of Assertions in StringUtilsTest.java.

c8aac1a

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Fixed indentation. Moved to 2 space indents to replace 4 space indents.

44e869b

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Put static imports before regular imports. Maintained alphabetical or…

56c737a

…der in StringUtilsTest.java Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Added case for double single quote which converts to a single single …

2454d90

…quote. Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Added new line at end of StringUtilsTest.java and added space between…

1cc549b

… static and non-static imports Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Replaced statement lambda with expression lambda.

621c689

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Allows for quotes(") and backticks(`) to be around strings in result.

70dada3

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Added additional test cases to unquoteTest() in StringUtilsTest.java.

bf4edec

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Added additional test cases to unquoteTest() in StringUtilsTest.java.…

6c8eb88

… Added case for double, double quote being converted to single, double quote. Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Made double single and double double quotes only reduce down to a sin…

1f2ab45

…gle of their respective type when the string is quoted by the same type. Added some unit tests to test for that condition in StringUtilsTest.java. Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Fixed whitespace found before comma in StringUtilsTest.java.

5d0ca9a

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Added more tests. Removed formatting changes in OpenSearchExecutionEn…

d0e0468

…gine.java. Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Reverting ProjectOperator.java.

907f56f

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Resolved issue with overlapping replacement of quotes (double and sin…

a144535

…gle). Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Moved replace periods to new line.

eae2092

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Changed StringUtils.java to use String Builder. Updated some tests.

8e213d6

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Changed over to fully use string builder and works with escape as wel…

c4e9b88

…l as double quotes. Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Fixed formatting of StringUtilsTest.java, removed code in comments in…

aa58736

… StringUtils.java. Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Double quote implemented to convert to single quote when string is su…

d67dd3d

…rrounded by same string. Consistent with MySQL standard. Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Cleaned up redundant code from StringUtils.java.

2de1495

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Removed IT test added.

4bb7a8d

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

MitchellGale added 7 commits July 22, 2022 08:43

Moved whichquote inside unquote text. Removed large portion of code f…

2899ee7

…rom if statement to break off for a return. Added unit test, removed unit tests for whichquote function. Removed whichquote function. Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Added some additional unit tests. Added branch for case of quoted by …

401d817

…back tick to skip over String Builder. Fixed issue with strings passed to unquotetext being too small casuing array bounds access issue. Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Fixed test case expected.

16927e8

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Reverting changes to OpenSearchExecutionEngine.java.

d887809

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Reverting changes to ProjectOperator.java.java.

e627e5e

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Added test cases in literals.txt.

1fadb92

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Added back tick and moved String builder declaration down.

1a7c927

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

MitchellGale dismissed joshuali925’s stale review via 1a7c927 July 22, 2022 15:44

MitchellGale force-pushed the Integ-DoubleQuote branch from f1e018c to 1a7c927 Compare July 22, 2022 15:44

Added better description regarding unquoteText function in regards to…

c799c6a

… double quote escape implementation. Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Merge branch 'main' of https://github.com/opensearch-project/sql into…

e7dd5a1

… Integ-DoubleQuote

ps48 pushed a commit to ps48/sql that referenced this pull request Jul 28, 2022

Collected fixes related to bugs #280, #302, opensearch-project#783, o…

646ce3c

…pensearch-project#696 and 317 all together. (opensearch-project#479) Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

Fixed line length of comments to not exceed 100 characters

ba86654

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

joshuali925 approved these changes Aug 1, 2022

View reviewed changes

mengweieric approved these changes Aug 1, 2022

View reviewed changes

joshuali925 merged commit 68663d1 into opensearch-project:main Aug 2, 2022

MaxKsyunz added enhancement New feature or request SQL labels Sep 27, 2022

Yury-Fridlyand mentioned this pull request Oct 14, 2022

[BUG] LIKE escape special characters with '\' not working #779

Open

dai-chen mentioned this pull request Dec 15, 2022

Unable to escape characters for string literals[BUG] #297

Closed

Yury-Fridlyand deleted the Integ-DoubleQuote branch January 12, 2023 19:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Two single or double quote escapes single or double quote when string is surrounded by same type of quote #696

Two single or double quote escapes single or double quote when string is surrounded by same type of quote #696

MitchellGale commented Jul 15, 2022

joshuali925 left a comment

penghuo commented Jul 21, 2022

codecov-commenter commented Jul 22, 2022 •

edited

Loading

Yury-Fridlyand commented Jul 22, 2022

dai-chen commented Aug 2, 2022

Two single or double quote escapes single or double quote when string is surrounded by same type of quote #696

Two single or double quote escapes single or double quote when string is surrounded by same type of quote #696

Conversation

MitchellGale commented Jul 15, 2022

Description

Issues Resolved

Check List

joshuali925 left a comment

Choose a reason for hiding this comment

penghuo commented Jul 21, 2022

codecov-commenter commented Jul 22, 2022 • edited Loading

Codecov Report

Yury-Fridlyand commented Jul 22, 2022

dai-chen commented Aug 2, 2022

codecov-commenter commented Jul 22, 2022 •

edited

Loading