Fix coredump for update if encountering bad format row #2024

dangleptr · 2020-03-30T12:06:07Z

Currently, there are so many CHECK and LOG(FATAL) inside RowReader, so if encountering bad format row (For example, upgrade the cluster), it will core dump.

Now, instead crash, storaged report TAG/EDGE NOT FOUND to graphd.

close #2016

bright-starry-sky

Well done! LGTM.

bright-starry-sky · 2020-03-30T14:19:52Z

The balance_integration_test test case failing as below. because port was used.

I0330 20:34:46.680209  8826 NebulaStore.cpp:173] Init data from partManager for [127.0.0.1:13468]
197
I0330 20:34:46.680218  8826 NebulaStore.cpp:188] Register handler...
198
I0330 20:34:46.926753  8886 HBProcessor.cpp:31] Receive heartbeat from [127.0.0.1:13468]
199
I0330 20:34:47.519037  8886 HBProcessor.cpp:31] Receive heartbeat from [127.0.0.1:13468]
200
I0330 20:34:47.553725  8886 HBProcessor.cpp:31] Receive heartbeat from [127.0.0.1:21586]
201
I0330 20:34:47.680382  8826 TestUtils.h:422] Create real SchemaManager and IndexManager
202
E0330 20:34:47.685719  9626 ThriftServer.cpp:440] Got an exception while setting up the server: 98failed to bind to async server socket: 0.0.0.0:13468: Address already in use
203
terminate called after throwing an instance of 'std::system_error'
204
  what():  98failed to bind to async server socket: 0.0.0.0:13468: Address already in use
205
*** Aborted at 1585571687 (unix time) try "date -d @1585571687" if you are using GNU date ***
206
PC: @     0x7fa8fb4ee337 __GI_raise

darionyaphet

LGTM

panda-sheep

Excellent，LGTM.
There is one tiny thing to note, see internal comment.

panda-sheep · 2020-03-31T01:50:11Z

src/storage/test/UpdateEdgeTest.cpp

+ PartitionID partId = 0;
+ VertexID srcId = 10;
+ VertexID dstId = 11;
+ // src = 1, edge_type = 101, ranking = 0, dst = 10001


update src = 10 , dst = 11 or remove the comment ？

Thanks for reminding. Let's me take care of it in next pr.

dangleptr · 2020-03-31T02:27:10Z

The balance_integration_test test case failing as below. because port was used.

I0330 20:34:46.680209  8826 NebulaStore.cpp:173] Init data from partManager for [127.0.0.1:13468]
197
I0330 20:34:46.680218  8826 NebulaStore.cpp:188] Register handler...
198
I0330 20:34:46.926753  8886 HBProcessor.cpp:31] Receive heartbeat from [127.0.0.1:13468]
199
I0330 20:34:47.519037  8886 HBProcessor.cpp:31] Receive heartbeat from [127.0.0.1:13468]
200
I0330 20:34:47.553725  8886 HBProcessor.cpp:31] Receive heartbeat from [127.0.0.1:21586]
201
I0330 20:34:47.680382  8826 TestUtils.h:422] Create real SchemaManager and IndexManager
202
E0330 20:34:47.685719  9626 ThriftServer.cpp:440] Got an exception while setting up the server: 98failed to bind to async server socket: 0.0.0.0:13468: Address already in use
203
terminate called after throwing an instance of 'std::system_error'
204
  what():  98failed to bind to async server socket: 0.0.0.0:13468: Address already in use
205
*** Aborted at 1585571687 (unix time) try "date -d @1585571687" if you are using GNU date ***
206
PC: @     0x7fa8fb4ee337 __GI_raise

Thanks.

@yixinlu

* Upgrade rocksdb to 6.7.3 (#1948) Co-authored-by: heng <heng.chen@vesoft.com> Co-authored-by: yaphet <darion.wang@vesoft.com> * Detect OS type and exit if not built in Linux (#1951) * Detect OS type and exit if not built in Linux Signed-off-by: JaySon-Huang <jayson.hjs@gmail.com> * Disable for FreeBSD Signed-off-by: JaySon-Huang <jayson.hjs@gmail.com> * [doc] user-permission-mng (#1929) * alter user & change psw * built in roles * cbs comments * fix comments * update table * addenable_authorize doc * [doc] ttl doc modification (#1939) * ttl doc modification * fix comments * yt comments * silver days comments * timestamp update * Let action not fail fast. (#1909) * Let action not fail fast. * Disable the ci will product artifact. * Remove removePrefix interface in kvstore (#1953) Co-authored-by: heng <heng.chen@vesoft.com> Co-authored-by: yaphet <darion.wang@vesoft.com> * using admin client in MetaServiceHandler (#1864) * [doc] fix typo (#1954) * [doc]show statements (#1924) * [doc]customized dir (#1906) * Fixed a bug caused by not filtering the versions of edges (#1962) * Fixed a bug caused by not filtering the versions of edges when executing "go from vid over *". * address CPWstatic's comment. * Modify the listEdges interface and add parameters to directly return all latest versions of edges. * address laura's comment. Co-authored-by: dutor <440396+dutor@users.noreply.github.com> * [doc]modify job manger doc (#1957) * modify job manger doc * add flush Co-authored-by: dutor <440396+dutor@users.noreply.github.com> * Fix delete verticse (#1970) * Fix the read performance issue after delete range with rocksdb-5.15.10 (#1973) * Fix the read performace issue after delete range with rocksdb-5.15.10 * Fix the move issue * fix the move issue * Rebase and address bright-starry-sky's comments Co-authored-by: heng <heng.chen@vesoft.com> * Remove the LOG(FATAL) when encounting the unknown log type (#1978) * 1,Improve customer friendliness; 2, fixed typo error (#1980) * upgrade spark download address (#1981) Co-authored-by: dutor <440396+dutor@users.noreply.github.com> * fix-typo-in-doc (#1976) Co-authored-by: dutor <440396+dutor@users.noreply.github.com> * modify commands in build doc (#1969) Co-authored-by: dutor <440396+dutor@users.noreply.github.com> * modify log level (#1945) Co-authored-by: dangleptr <37216992+dangleptr@users.noreply.github.com> Co-authored-by: dutor <440396+dutor@users.noreply.github.com> * use bloom filter, add event listener to collect stats of compaction, improve update rocksdb options (#1959) * open bloom filter in default conf, add event listener to collect stats of compaction * imporve update rocksdb configs * fix ut error Co-authored-by: yaphet <darion.wang@vesoft.com> * Update the interface atomicOp to support empty string (#2002) Co-authored-by: heng <heng.chen@vesoft.com> Co-authored-by: Doodle <13706157+critical27@users.noreply.github.com> * update to rc4 (#1968) Co-authored-by: dutor <440396+dutor@users.noreply.github.com> * Fixed if you execute "go from xx over *", and when there is no edge under a space, (#1983) an unreasonable error message is returned (SpaceId `xxx 'is nonexistent) Co-authored-by: dutor <440396+dutor@users.noreply.github.com> * [doc]pr-commit-msg (#1993) * pr-commit-msg * add examples Co-authored-by: dutor <440396+dutor@users.noreply.github.com> * doc:modify ttl query (#2006) Co-authored-by: dutor <440396+dutor@users.noreply.github.com> * Let merge coverage report faster by fastcov instead of lcov. (#1943) Co-authored-by: dutor <440396+dutor@users.noreply.github.com> * Auto check keyword (#1922) * Supports the detection keyword, name_label supports string * add tests * address Shylock's comment * modify to add reserved keyword Co-authored-by: dutor <440396+dutor@users.noreply.github.com> * Fix coredump for update if encountering bad format row (#2024) Co-authored-by: heng <heng.chen@vesoft.com> Co-authored-by: bright-starry-sky <56461666+bright-starry-sky@users.noreply.github.com> * fixed loop depend problem for compile link (#1995) * fixed loop depend problem * fixed error for clang9 and gcc9 * Addressed laura-ding's comment * Check all reader is null or not before using it (#2029) * fix go without yield return errror result (#2012) * fix go without yield return errror result * fix format * address dangleptr's comment * address issue change TTL_DURATION not negative * add onlyStructure is true test case Co-authored-by: bright-starry-sky <56461666+bright-starry-sky@users.noreply.github.com> * fixed bug for delete vertex without edge (#2001) * When deleting a vertex, if the edge does not exist, then we should continue to delete the vertex instead of reporting an error. * added the testcase for delete vertex without edge. * add alias for lookup (#2000) * fix bug 1990/1991 (#2007) * 1. meta client sub mit job only to leader 2. sort jobs in desc id order * now show jobs return in desc order, adjust UT Co-authored-by: dangleptr <37216992+dangleptr@users.noreply.github.com> * upload package to oss (#1972) * upload oss * update * Remove redundant * address @yixinlu's comment Co-authored-by: dutor <440396+dutor@users.noreply.github.com> Co-authored-by: dangleptr <37216992+dangleptr@users.noreply.github.com> Co-authored-by: heng <heng.chen@vesoft.com> Co-authored-by: yaphet <darion.wang@vesoft.com> Co-authored-by: JaySon <jayson.hjs@gmail.com> Co-authored-by: Amber Zhang <42762957+Amber1990Zhang@users.noreply.github.com> Co-authored-by: Shylock Hg <33566796+Shylock-Hg@users.noreply.github.com> Co-authored-by: Simon Liu <331435+monadbobo@users.noreply.github.com> Co-authored-by: dutor <440396+dutor@users.noreply.github.com> Co-authored-by: bright-starry-sky <56461666+bright-starry-sky@users.noreply.github.com> Co-authored-by: Liuxue-Yang <50105172+Liuxue-Yang@users.noreply.github.com> Co-authored-by: laura-ding <48548375+laura-ding@users.noreply.github.com> Co-authored-by: Doodle <13706157+critical27@users.noreply.github.com> Co-authored-by: panda-sheep <59197347+panda-sheep@users.noreply.github.com> Co-authored-by: liuyu85cn <52276794+liuyu85cn@users.noreply.github.com>

Co-authored-by: heng <heng.chen@vesoft.com> Co-authored-by: bright-starry-sky <56461666+bright-starry-sky@users.noreply.github.com>

Co-authored-by: Doodle <13706157+critical27@users.noreply.github.com>

Fix coredump for update if encountering bad format row

9ba3cc1

dangleptr requested review from monadbobo, darionyaphet, zhangguoqing, CPWstatic, critical27, Shylock-Hg, liuyu85cn, bright-starry-sky and panda-sheep March 30, 2020 12:06

dangleptr added the ready-for-testing PR: ready for the CI test label Mar 30, 2020

bright-starry-sky approved these changes Mar 30, 2020

View reviewed changes

Merge branch 'master' into report

d88691e

darionyaphet approved these changes Mar 30, 2020

View reviewed changes

panda-sheep approved these changes Mar 31, 2020

View reviewed changes

dangleptr merged commit 4e434a5 into vesoft-inc:master Mar 31, 2020

dangleptr mentioned this pull request Mar 31, 2020

Check reader is null or not before using it #2029

Merged

jude-zhu added the rc4-cherrypick label Mar 31, 2020

yixinglu pushed a commit to yixinglu/nebula that referenced this pull request Jan 31, 2023

forbid string index length <= 0, ent issue 1685 (vesoft-inc#2024)

b2b0d85

Co-authored-by: Doodle <13706157+critical27@users.noreply.github.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix coredump for update if encountering bad format row #2024

Fix coredump for update if encountering bad format row #2024

dangleptr commented Mar 30, 2020 •

edited

Loading

bright-starry-sky left a comment

bright-starry-sky commented Mar 30, 2020

darionyaphet left a comment

panda-sheep left a comment

panda-sheep Mar 31, 2020 •

edited

Loading

dangleptr Mar 31, 2020

dangleptr commented Mar 31, 2020

Fix coredump for update if encountering bad format row #2024

Fix coredump for update if encountering bad format row #2024

Conversation

dangleptr commented Mar 30, 2020 • edited Loading

bright-starry-sky left a comment

Choose a reason for hiding this comment

bright-starry-sky commented Mar 30, 2020

darionyaphet left a comment

Choose a reason for hiding this comment

panda-sheep left a comment

Choose a reason for hiding this comment

panda-sheep Mar 31, 2020 • edited Loading

Choose a reason for hiding this comment

dangleptr Mar 31, 2020

Choose a reason for hiding this comment

dangleptr commented Mar 31, 2020

dangleptr commented Mar 30, 2020 •

edited

Loading

panda-sheep Mar 31, 2020 •

edited

Loading