Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(load): fix mysqld crash when loaded data column contains double enclosed char. (#1263) #1268

Merged
merged 2 commits into from
Feb 13, 2023

Conversation

lujiashun
Copy link

@lujiashun lujiashun commented Feb 2, 2023

Summary about this PR

Issue Number: close #1263

  1. crash reason is the loader column number each time is not 65535 but report 65535;
    occurs when using limit clause or duplicate key;
  2. support double enclosed char, ref:https://dev.mysql.com/doc/refman/5.7/en/load-data.html

Tests Check List

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Changelog

  • New Feature
  • Bug Fix
  • Performance Improvement
  • Build/Testing/CI/CD
  • Documentation
  • Not for changelog (changelog entry is not required)

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features

@mergify
Copy link
Contributor

mergify bot commented Feb 2, 2023

Thanks for the contribution!
I have applied any labels matching special text in your PR Changelog.

Please review the labels and make any necessary changes.

@mergify mergify bot added the PR-bug bug for pull request label Feb 2, 2023
@lujiashun lujiashun self-assigned this Feb 2, 2023
@lujiashun lujiashun changed the title fix(load): fix mysqld crash when columns contains double enclosed char. (#1263) fix(load): fix mysqld crash when loaded data column contains double enclosed char. (#1263) Feb 2, 2023
++ptr;
}
} else if (size == 2) {
--search_end;
while (ptr < search_end && (*ptr != *c_pattern || ptr[1] != c_pattern[1])) {
if (escape_char_ && *ptr == escape_char_)
ptr += 2;
else
else if (string_qualifier_ && *ptr == string_qualifier_ && ptr + 1 < search_end &&
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this piece of code is repeated 4 times, how about add a new function for it?

(string_qualifier_ && *ptr == string_qualifier_ && ptr + 1 < search_end &&
*(ptr + 1) == string_qualifier_)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed it.

@lujiashun lujiashun force-pushed the fix-1263-stonedb5.7 branch from 509db0a to 0423e1a Compare February 3, 2023 04:01
@codecov
Copy link

codecov bot commented Feb 3, 2023

Codecov Report

Base: 43.28% // Head: 43.31% // Increases project coverage by +0.02% 🎉

Coverage data is based on head (caade14) compared to base (07a3d72).
Patch coverage: 46.66% of modified lines in pull request are covered.

Additional details and impacted files
@@                 Coverage Diff                 @@
##           stonedb-5.7-dev    #1268      +/-   ##
===================================================
+ Coverage            43.28%   43.31%   +0.02%     
===================================================
  Files                 1830     1830              
  Lines               396156   396171      +15     
===================================================
+ Hits                171483   171606     +123     
+ Misses              224673   224565     -108     
Impacted Files Coverage Δ
storage/tianmu/loader/load_parser.cpp 69.91% <ø> (+6.19%) ⬆️
storage/tianmu/loader/parsing_strategy.cpp 67.36% <46.66%> (+1.22%) ⬆️
storage/innobase/include/ut0mutex.ic 52.38% <0.00%> (-19.05%) ⬇️
storage/innobase/row/row0umod.cc 62.50% <0.00%> (-4.17%) ⬇️
storage/innobase/include/ib0mutex.h 71.45% <0.00%> (-0.75%) ⬇️
strings/ctype-simple.c 60.86% <0.00%> (-0.51%) ⬇️
storage/innobase/btr/btr0sea.cc 74.81% <0.00%> (-0.37%) ⬇️
storage/innobase/buf/buf0buf.cc 45.90% <0.00%> (-0.32%) ⬇️
storage/tianmu/core/sorter3.cpp 42.94% <0.00%> (-0.31%) ⬇️
storage/innobase/row/row0sel.cc 63.25% <0.00%> (-0.18%) ⬇️
... and 24 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@lujiashun lujiashun force-pushed the fix-1263-stonedb5.7 branch from 0423e1a to da4a8eb Compare February 3, 2023 04:29
@@ -0,0 +1,2 @@
"765893","37","27","44221","2021-06-28 02:01:16","计算售罄率出错:ORA-04036: PGA memory used by the instance exceeds PGA_AGGREGATE_LIMITORA-06512: at ""BOSNDS3.O2O_STORESALERATE_SET"", line 49
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what will happens? """ xxx """, and """" xxxx """"

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add these test case . Please check the MTR result file or datafile(issue1263-2.txt and issue1263-3.txt).

  1. double enclosed char will be interpred as a normal char;
  2. singe enclosed char before field-terminated string will be interpred as enclose char
  3. sing enclosed char with non-filed-termniated string will be interpreted as a normal char;

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PTAL, @RingsC

@lujiashun lujiashun force-pushed the fix-1263-stonedb5.7 branch 2 times, most recently from 87af55d to 8ddcfb3 Compare February 3, 2023 13:26
…r. (stoneatom#1263)

[summary]
1. crash reason is the loader column number each time is not 65535 but report 65535;
   occurs when using limit clause or duplicate key;
2. support double enclosed char, ref:https://dev.mysql.com/doc/refman/5.7/en/load-data.html
@lujiashun lujiashun force-pushed the fix-1263-stonedb5.7 branch from 8ddcfb3 to b08ae17 Compare February 3, 2023 13:49
@lujiashun lujiashun requested a review from DandreChen February 10, 2023 03:46
Copy link
Collaborator

@DandreChen DandreChen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mergify mergify bot merged commit ad4e0fa into stoneatom:stonedb-5.7-dev Feb 13, 2023
@Nliver Nliver added this to the StoneDB_5.7_v1.0.3 milestone Apr 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
PR-bug bug for pull request
Projects
Development

Successfully merging this pull request may close these issues.

bug: The instance crashed after I execute a statement that includes a condition to load data.
5 participants