Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bug: The instance crashed after I execute a statement that includes a condition to load data. #1263

Closed
3 tasks done
haitaoguan opened this issue Feb 1, 2023 · 4 comments · Fixed by #1268
Closed
3 tasks done

bug: The instance crashed after I execute a statement that includes a condition to load data. #1263

haitaoguan opened this issue Feb 1, 2023 · 4 comments · Fixed by #1268
Assignees
@lujiashun
Labels
A-bug Something isn't working B-loader load data to stonedb prio: high High priority
Milestone
StoneDB_5.7_v1.0.3

Comments

@haitaoguan
Copy link
Collaborator

haitaoguan commented Feb 1, 2023 •
edited
Loading

Have you read the Contributing Guidelines on issues?

Please confirm if bug report does NOT exists already ?

  • I confirm there is no existing issue for this

Describe the problem

CREATE TABLE AD_PINSTANCE_LOG
   (ID bigint, 
	AD_CLIENT_ID bigint, 
	AD_ORG_ID bigint, 
	AD_PINSTANCE_ID bigint, 
	P_DATE DATETIME DEFAULT null, 
	P_MSG VARCHAR(255), 
	OWNERID bigint, 
	MODIFIERID bigint, 
	CREATIONDATE DATETIME, 
	MODIFIEDDATE DATETIME, 
	ISACTIVE CHAR(1) DEFAULT 'Y' NOT NULL, 
	 PRIMARY KEY (ID));

load data infile '/tmp/AD_PINSTANCE_LOG' into table AD_PINSTANCE_LOG
FIELDS TERMINATED BY ',' 
ENCLOSED BY '"' 
LINES TERMINATED BY '\n'
(
@ID,
@AD_CLIENT_ID,
@AD_ORG_ID,
@AD_PINSTANCE_ID,
@P_DATE,
@P_MSG,
@OWNERID,
@MODIFIERID,
@CREATIONDATE,
@MODIFIEDDATE,
@ISACTIVE
)
set
ID=NULLif(@ID,''),
AD_CLIENT_ID=NULLif(@AD_CLIENT_ID,''),
AD_ORG_ID=NULLif(@AD_ORG_ID,''),
AD_PINSTANCE_ID=NULLif(@AD_PINSTANCE_ID,''),
P_DATE=NULLif(@P_DATE,''),
P_MSG=NULLif(@P_MSG,''),
OWNERID=NULLif(@OWNERID,''),
MODIFIERID=NULLif(@MODIFIERID,''),
CREATIONDATE=NULLif(@CREATIONDATE,''),
MODIFIEDDATE=NULLif(@MODIFIEDDATE,''),
ISACTIVE=NULLif(@ISACTIVE,'')
;
ERROR 2013 (HY000): Lost connection to MySQL server during query

Expected behavior

No response

How To Reproduce

No response

Environment

build information as follow:
Repository address: https://github.com/stoneatom/stonedb.git:stonedb-5.7-dev
Branch name: stonedb-5.7-dev
Last commit ID: 29e3f7c
Last commit time: Date: Thu Jan 12 17:58:24 2023 +0800
Build time: Date: Mon Jan 16 08:21:32 CST 2023

Are you interested in submitting a PR to solve the problem?

  • Yes, I will!
@haitaoguan haitaoguan added the A-bug Something isn't working label Feb 1, 2023
@wisehead wisehead added this to the stonedb_5.7_v1.0.3 milestone Feb 1, 2023
@wisehead wisehead added B-loader load data to stonedb prio: high High priority labels Feb 1, 2023
@lujiashun
Copy link

ACK

@haitaoguan haitaoguan changed the title bug: The instance crashed because the load data statement contained a conditional judgment. bug: The instance crashed after I execute a statement that includes a condition to load data. Feb 1, 2023
@lujiashun
Copy link

lujiashun commented Feb 1, 2023 •
edited
Loading

the data ptr life-cycle in load is as below:

Thread 26 "loader[0]" hit Breakpoint 1, Tianmu::core::DPN::SetPackPtr (this=0x7fd3424f6210, v=0) at /data/codebase/stonedb/storage/tianmu/core/dpn.h:83
83        void SetPackPtr(uint64_t v) { tagged_ptr.store(v); }
$183 = 0
#0  Tianmu::core::DPN::SetPackPtr (this=0x7fd3424f6210, v=0) at /data/codebase/stonedb/storage/tianmu/core/dpn.h:83
#1  0x0000000002fab590 in Tianmu::core::ColumnShare::init_dpn (this=0x7fd32801ce00, dpn=..., xid=..., from=0x0) at /data/codebase/stonedb/storage/tianmu/core/column_share.cpp:215
#2  0x0000000002fab6fc in Tianmu::core::ColumnShare::alloc_dpn (this=0x7fd32801ce00, xid=..., from=0x0) at /data/codebase/stonedb/storage/tianmu/core/column_share.cpp:226
[Switching to Thread 0x7fd5dffff700 (LWP 10495)]

Thread 26 "loader[0]" hit Breakpoint 1, Tianmu::core::DPN::SetPackPtr (this=0x7fd3424f6210, v=422031875115760) at /data/codebase/stonedb/storage/tianmu/core/dpn.h:83
83        void SetPackPtr(uint64_t v) { tagged_ptr.store(v); }
$192 = 422031875115760
#0  Tianmu::core::DPN::SetPackPtr (this=0x7fd3424f6210, v=422031875115760) at /data/codebase/stonedb/storage/tianmu/core/dpn.h:83
#1  0x0000000002d34a42 in Tianmu::core::TianmuAttr::LoadDataPackN (this=0x7fd32898b280, pi=6, nvs=0x7fd328bbd1f0) at /data/codebase/stonedb/storage/tianmu/core/tianmu_attr.cpp:963
#2  0x0000000002d343eb in Tianmu::core::TianmuAttr::LoadData (this=0x7fd32898b280, nvs=0x7fd328bbd1f0, conn_info=0x7fd32801ec00) at /data/codebase/stonedb/storage/tianmu/core/tianmu_attr.cpp:881
[Switching to Thread 0x7fd5e77fe700 (LWP 10496)]

Thread 26 "loader[0]" hit Breakpoint 1, Tianmu::core::DPN::SetPackPtr (this=0x7fd3424f6210, v=0) at /data/codebase/stonedb/storage/tianmu/core/dpn.h:83
83        void SetPackPtr(uint64_t v) { tagged_ptr.store(v); }
$200 = 0
#0  Tianmu::core::DPN::SetPackPtr (this=0x7fd3424f6210, v=0) at /data/codebase/stonedb/storage/tianmu/core/dpn.h:83
#1  0x0000000002d3456d in Tianmu::core::TianmuAttr::LoadData (this=0x7fd32898b280, nvs=0x7fd328bbd1f0, conn_info=0x7fd32801ec00) at /data/codebase/stonedb/storage/tianmu/core/tianmu_attr.cpp:902
#2  0x0000000002d88ca3 in std::__invoke_impl<void, void (Tianmu::core::TianmuAttr::*&)(Tianmu::loader::ValueCache*, Tianmu::core::Transaction*), Tianmu::core::TianmuAttr<Tianmu::loader::ValueCache*&, Tianmu::core::Transaction*&>*&> (__f=@0x7fd329011488: (void (Tianmu::core::TianmuAttr::*)(Tianmu::core::TianmuAttr * const, Tianmu::loader::ValueCache *, Tianmu::core::Transaction *)) 0x2d34368 <Tianmu::core::TianmuAttr::LoadData(Tianmu::loader::ValueCache*, Tianmu::core::Transaction*)>, __t=@0x7fd3290114a8: 0x7fd32898b280) at /opt/rh/devtoolset-9/root/usr/include/c++/9/bits/invoke.h:73
[Switching to Thread 0x7fd609ffb700 (LWP 10491)]

@lujiashun
Copy link

lujiashun commented Feb 1, 2023 •
edited
Loading

if not the last pack, the insert value number should be 65536, otherwise it will crash. In this scenario. the last but one is 65534

--
Thread 33 "loader[7]" hit Breakpoint 2, Tianmu::core::TianmuAttr::LoadDataPackN (this=0x7f3c1898a880, pi=5, nvs=0x7f3c195e9148) at /data/codebase/stonedb/storage/tianmu/core/tianmu_attr.cpp:925
925       if (load_nulls == load_values && (dpn.numOfRecords == 0 || dpn.NullOnly())) {
$229 = 1
$230 = 65536
[Switching to Thread 0x7f3f01ffb700 (LWP 12135)]

Thread 27 "loader[1]" hit Breakpoint 2, Tianmu::core::TianmuAttr::LoadDataPackN (this=0x7f3c1897f900, pi=5, nvs=0x7f3c195e90a0) at /data/codebase/stonedb/storage/tianmu/core/tianmu_attr.cpp:925
925       if (load_nulls == load_values && (dpn.numOfRecords == 0 || dpn.NullOnly())) {
$231 = 0
$232 = 65536
[Switching to Thread 0x7f3ed77fe700 (LWP 12140)]

Thread 32 "loader[6]" hit Breakpoint 2, Tianmu::core::TianmuAttr::LoadDataPackN (this=0x7f3c1898be80, pi=5, nvs=0x7f3c195e95e0) at /data/codebase/stonedb/storage/tianmu/core/tianmu_attr.cpp:925
925       if (load_nulls == load_values && (dpn.numOfRecords == 0 || dpn.NullOnly())) {
$233 = 8
$234 = 65536
[Switching to Thread 0x7f3f00ff9700 (LWP 12137)]

--
Thread 29 "loader[3]" hit Breakpoint 2, Tianmu::core::TianmuAttr::LoadDataPackN (this=0x7f3c1898c180, pi=5, nvs=0x7f3c195e9688) at /data/codebase/stonedb/storage/tianmu/core/tianmu_attr.cpp:925
925       if (load_nulls == load_values && (dpn.numOfRecords == 0 || dpn.NullOnly())) {
$239 = 9
$240 = 65536
[Switching to Thread 0x7f3f027fc700 (LWP 12134)]

--
Thread 30 "loader[4]" hit Breakpoint 2, Tianmu::core::TianmuAttr::LoadDataPackN (this=0x7f3c1898ab80, pi=6, nvs=0x7f3c195e91f0) at /data/codebase/stonedb/storage/tianmu/core/tianmu_attr.cpp:925
925       if (load_nulls == load_values && (dpn.numOfRecords == 0 || dpn.NullOnly())) {
$255 = 2
$256 = 65534
[Switching to Thread 0x7f3f027fc700 (LWP 12134)]

Thread 26 "loader[0]" hit Breakpoint 2, Tianmu::core::TianmuAttr::LoadDataPackN (this=0x7f3c1898b880, pi=6, nvs=0x7f3c195e9490) at /data/codebase/stonedb/storage/tianmu/core/tianmu_attr.cpp:925
925       if (load_nulls == load_values && (dpn.numOfRecords == 0 || dpn.NullOnly())) {
$257 = 6
$258 = 65534
[Switching to Thread 0x7f3f00ff9700 (LWP 12137)]

--
Thread 33 "loader[7]" hit Breakpoint 2, Tianmu::core::TianmuAttr::LoadDataPackN (this=0x7f3c1898bb80, pi=6, nvs=0x7f3c195e9538) at /data/codebase/stonedb/storage/tianmu/core/tianmu_attr.cpp:925
925       if (load_nulls == load_values && (dpn.numOfRecords == 0 || dpn.NullOnly())) {
$261 = 7
$262 = 65534
[Switching to Thread 0x7f3ecffff700 (LWP 12139)]

Thread 31 "loader[5]" hit Breakpoint 2, Tianmu::core::TianmuAttr::LoadDataPackN (this=0x7f3c1898b200, pi=6, nvs=0x7f3c195e9340) at /data/codebase/stonedb/storage/tianmu/core/tianmu_attr.cpp:925
925       if (load_nulls == load_values && (dpn.numOfRecords == 0 || dpn.NullOnly())) {
$263 = 4
$264 = 65534
[Switching to Thread 0x7f3f01ffb700 (LWP 12135)]

Thread 27 "loader[1]" hit Breakpoint 2, Tianmu::core::TianmuAttr::LoadDataPackN (this=0x7f3c1898a880, pi=6, nvs=0x7f3c195e9148) at /data/codebase/stonedb/storage/tianmu/core/tianmu_attr.cpp:925
925       if (load_nulls == load_values && (dpn.numOfRecords == 0 || dpn.NullOnly())) {
$265 = 1
$266 = 65534
[Switching to Thread 0x7f3f017fa700 (LWP 12136)]

Thread 28 "loader[2]" hit Breakpoint 2, Tianmu::core::TianmuAttr::LoadDataPackN (this=0x7f3c1897f900, pi=6, nvs=0x7f3c195e90a0) at /data/codebase/stonedb/storage/tianmu/core/tianmu_attr.cpp:925
925       if (load_nulls == load_values && (dpn.numOfRecords == 0 || dpn.NullOnly())) {
$267 = 0
$268 = 65534
[Switching to Thread 0x7f3ed77fe700 (LWP 12140)]

Thread 32 "loader[6]" hit Breakpoint 2, Tianmu::core::TianmuAttr::LoadDataPackN (this=0x7f3c1898ae80, pi=6, nvs=0x7f3c195e9298) at /data/codebase/stonedb/storage/tianmu/core/tianmu_attr.cpp:925
925       if (load_nulls == load_values && (dpn.numOfRecords == 0 || dpn.NullOnly())) {
$269 = 3
$270 = 65534
[Switching to Thread 0x7f3ed7fff700 (LWP 12138)]

--
Thread 30 "loader[4]" hit Breakpoint 2, Tianmu::core::TianmuAttr::LoadDataPackN (this=0x7f3c1898be80, pi=6, nvs=0x7f3c195e95e0) at /data/codebase/stonedb/storage/tianmu/core/tianmu_attr.cpp:925
925       if (load_nulls == load_values && (dpn.numOfRecords == 0 || dpn.NullOnly())) {
$275 = 8
$276 = 65534
[Switching to Thread 0x7f3f01ffb700 (LWP 12135)]

--
Thread 26 "loader[0]" hit Breakpoint 2, Tianmu::core::TianmuAttr::LoadDataPackN (this=0x7f3c1898c180, pi=6, nvs=0x7f3c195e9688) at /data/codebase/stonedb/storage/tianmu/core/tianmu_attr.cpp:925
925       if (load_nulls == load_values && (dpn.numOfRecords == 0 || dpn.NullOnly())) {
$279 = 9
$280 = 65534
[Switching to Thread 0x7f3f01ffb700 (LWP 12135)]

--
Thread 27 "loader[1]" hit Breakpoint 2, Tianmu::core::TianmuAttr::LoadDataPackN (this=0x7f3c1898bb80, pi=6, nvs=0x7f3c195e9538) at /data/codebase/stonedb/storage/tianmu/core/tianmu_attr.cpp:925
925       if (load_nulls == load_values && (dpn.numOfRecords == 0 || dpn.NullOnly())) {
$289 = 7
$290 = 2
[Switching to Thread 0x7f3f027fc700 (LWP 12134)]

--
Thread 28 "loader[2]" hit Breakpoint 2, Tianmu::core::TianmuAttr::LoadDataPackN (this=0x7f3c1897f900, pi=6, nvs=0x7f3c195e90a0) at /data/codebase/stonedb/storage/tianmu/core/tianmu_attr.cpp:925
925       if (load_nulls == load_values && (dpn.numOfRecords == 0 || dpn.NullOnly())) {
$297 = 0
$298 = 2

Thread 27 "loader[1]" received signal SIGSEGV, Segmentation fault.

@lujiashun
Copy link 

(gdb) n
794         vecInt.push_back(no_of_rows_returned);
1: no_of_rows_returned = 65536
(gdb) p value_buffers
$8 = std::vector of length 11, capacity 11 = {{data_ = 0x7ff1e6f31360, size_ = 524272, capacity_ = 576844, value_count_ = 65536, expected_size_ = 0,
    expected_null_ = false, values_ = std::vector of length 65534, capacity 65536 = {0, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128,
      136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256, 264, 272, 280, 288, 296, 304, 312, 320, 328, 336, 344, 352, 360, 368,
      376, 384, 392, 400, 408, 416, 424, 432, 440, 448, 456, 464, 472, 480, 488, 496, 504, 512, 520, 528, 536, 544, 552, 560, 568, 576, 584, 592, 600, 608,
      616, 624, 632, 640, 648, 656, 664, 672, 680, 688, 696, 704, 712, 720, 728, 736, 744, 752, 760, 768, 776, 784, 792, 800, 808, 816, 824, 832, 840, 848,
      856, 864, 872, 880, 888, 896, 904, 912, 920, 928, 936, 944, 952, 960, 968, 976, 984, 992, 1000, 1008, 1016, 1024, 1032, 1040, 1048, 1056, 1064, 1072,
      1080, 1088, 1096, 1104, 1112, 1120, 1128, 1136, 1144, 1152, 1160, 1168, 1176, 1184, 1192, 1200, 1208, 1216, 1224, 1232, 1240, 1248, 1256, 1264, 1272,
      1280, 1288, 1296, 1304, 1312, 1320, 1328, 1336, 1344, 1352, 1360, 1368, 1376, 1384, 1392, 1400, 1408, 1416, 1424, 1432, 1440, 1448, 1456, 1464, 1472,
      1480, 1488, 1496, 1504, 1512, 1520, 1528, 1536, 1544, 1552, 1560, 1568, 1576, 1584, 1592...},

lujiashun pushed a commit to lujiashun/stonedb that referenced this issue Feb 2, 2023
fix(load): fix mysqld crash when columns contains double enclosed cha…
66353af 
…r. (stoneatom#1263)

[summary]
1. crash reason is the loader column number each time is not 65535 but report 65535;
   occurs when using limit clause or duplicate key;
2. suppprt double enclosed char, ref:https://dev.mysql.com/doc/refman/5.7/en/load-data.html
lujiashun pushed a commit to lujiashun/stonedb that referenced this issue Feb 2, 2023
fix(load): fix mysqld crash when columns contains double enclosed cha…
509db0a 
…r. (stoneatom#1263)

[summary]
1. crash reason is the loader column number each time is not 65535 but report 65535;
   occurs when using limit clause or duplicate key;
2. support double enclosed char, ref:https://dev.mysql.com/doc/refman/5.7/en/load-data.html
@lujiashun lujiashun mentioned this issue Feb 2, 2023
Merged
14 tasks
lujiashun pushed a commit to lujiashun/stonedb that referenced this issue Feb 3, 2023
fix(load): fix mysqld crash when columns contains double enclosed cha…
0423e1a 
…r. (stoneatom#1263)

[summary]
1. crash reason is the loader column number each time is not 65535 but report 65535;
   occurs when using limit clause or duplicate key;
2. support double enclosed char, ref:https://dev.mysql.com/doc/refman/5.7/en/load-data.html
lujiashun pushed a commit to lujiashun/stonedb that referenced this issue Feb 3, 2023
fix(load): fix mysqld crash when columns contains double enclosed cha…
da4a8eb 
…r. (stoneatom#1263)

[summary]
1. crash reason is the loader column number each time is not 65535 but report 65535;
   occurs when using limit clause or duplicate key;
2. support double enclosed char, ref:https://dev.mysql.com/doc/refman/5.7/en/load-data.html
lujiashun pushed a commit to lujiashun/stonedb that referenced this issue Feb 3, 2023
fix(load): fix mysqld crash when columns contains double enclosed cha…
87af55d 
…r. (stoneatom#1263)

[summary]
1. crash reason is the loader column number each time is not 65535 but report 65535;
   occurs when using limit clause or duplicate key;
2. support double enclosed char, ref:https://dev.mysql.com/doc/refman/5.7/en/load-data.html
lujiashun pushed a commit to lujiashun/stonedb that referenced this issue Feb 3, 2023
fix(load): fix mysqld crash when columns contains double enclosed cha…
8ddcfb3 
…r. (stoneatom#1263)

[summary]
1. crash reason is the loader column number each time is not 65535 but report 65535;
   occurs when using limit clause or duplicate key;
2. support double enclosed char, ref:https://dev.mysql.com/doc/refman/5.7/en/load-data.html
lujiashun pushed a commit to lujiashun/stonedb that referenced this issue Feb 3, 2023
fix(load): fix mysqld crash when columns contains double enclosed cha…
b08ae17 
…r. (stoneatom#1263)

[summary]
1. crash reason is the loader column number each time is not 65535 but report 65535;
   occurs when using limit clause or duplicate key;
2. support double enclosed char, ref:https://dev.mysql.com/doc/refman/5.7/en/load-data.html
@mergify mergify bot closed this as completed in #1268 Feb 13, 2023
mergify bot pushed a commit that referenced this issue Feb 13, 2023
@mergify
fix(load): fix mysqld crash when columns contains double enclosed cha…
ad4e0fa 
…r. (#1263)

[summary]
1. crash reason is the loader column number each time is not 65535 but report 65535;
   occurs when using limit clause or duplicate key;
2. support double enclosed char, ref:https://dev.mysql.com/doc/refman/5.7/en/load-data.html
isredstar pushed a commit to isredstar/stonedb that referenced this issue Feb 15, 2023
@isredstar
fix(load): fix mysqld crash when columns contains double enclosed cha…
bc0d364 
…r. (stoneatom#1263)

[summary]
1. crash reason is the loader column number each time is not 65535 but report 65535;
   occurs when using limit clause or duplicate key;
2. support double enclosed char, ref:https://dev.mysql.com/doc/refman/5.7/en/load-data.html
mergify bot pushed a commit that referenced this issue Feb 27, 2023
@mergify
fix(load): fix mysqld crash when columns contains double enclosed cha…
a5bc0ee 
…r. (#1263)

[summary]
1. crash reason is the loader column number each time is not 65535 but report 65535;
   occurs when using limit clause or duplicate key;
2. support double enclosed char, ref:https://dev.mysql.com/doc/refman/5.7/en/load-data.html
konghaiya pushed a commit to konghaiya/stonedb that referenced this issue Mar 7, 2023
@konghaiya
fix(load): fix mysqld crash when columns contains double enclosed cha…
80a1c0a 
…r. (stoneatom#1263)

[summary]
1. crash reason is the loader column number each time is not 65535 but report 65535;
   occurs when using limit clause or duplicate key;
2. support double enclosed char, ref:https://dev.mysql.com/doc/refman/5.7/en/load-data.html
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@lujiashun lujiashun

Labels
A-bug Something isn't working B-loader load data to stonedb prio: high High priority
Projects
None yet
Milestone
StoneDB_5.7_v1.0.3
3 participants
@wisehead @haitaoguan @lujiashun