nexmark_chaos_q102_multiple: mem-table operation conflicts

[lmatz]

Describe the bug

nexmark_chaos_q102_multiple

seed = 1672241879729391739

thread '<unnamed>' panicked at 'mem-table operation conflicts! table_id: 1022, vnode: 157, key: OwnedRow([Some(Int32(3400)), Some(Int64(118305927388119043))]), prev: Insert(Ok(OwnedRow([Some(Int32(3400)), Some(Int64(118305927388119043))]))), new: Insert(Ok(OwnedRow([Some(Int32(3400)), Some(Int64(118305927388119043))])))', /risingwave/src/stream/src/common/table/state_table.rs:505:17
--
  | stack backtrace:
  | 0: rust_begin_unwind
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/std/src/panicking.rs:575:5
  | 1: core::panicking::panic_fmt
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/core/src/panicking.rs:64:14
  | 2: risingwave_stream::common::table::state_table::StateTable<S>::handle_mem_table_error
  | at /risingwave/src/stream/src/common/table/state_table.rs:505:17
  | 3: risingwave_stream::common::table::state_table::StateTable<S>::insert::{{closure}}
  | at /risingwave/src/stream/src/common/table/state_table.rs:534:33
  | 4: core::result::Result<T,E>::unwrap_or_else
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/core/src/result.rs:1504:23
  | 5: risingwave_stream::common::table::state_table::StateTable<S>::insert
  | at /risingwave/src/stream/src/common/table/state_table.rs:532:9
  | 6: risingwave_stream::executor::managed_state::join::JoinHashMap<K,S>::insert_row
  | at /risingwave/src/stream/src/executor/managed_state/join/mod.rs:427:9
  | 7: risingwave_stream::executor::hash_join::HashJoinExecutor<K,S,_>::eq_join_oneside::{{closure}}
  | at /risingwave/src/stream/src/executor/hash_join.rs:862:25
  | 8: <futures_async_stream::try_stream::from_generator::GenTryStream<G> as futures_core::stream::Stream>::poll_next
  | at /risingwave/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-async-stream-0.2.5/src/lib.rs:430:31
  | 9: risingwave_stream::executor::hash_join::HashJoinExecutor<K,S,_>::into_stream::{{closure}}
  | at /risingwave/src/stream/src/executor/hash_join.rs:591:5

To Reproduce

No response

Expected behavior

No response

Additional context

https://buildkite.com/risingwavelabs/pull-request/builds/14699#01855957-a784-446f-9bfa-a86d18629413

[BugenZhao]

Occurred frequently. 👀

https://buildkite.com/risingwavelabs/pull-request/builds/14717#01855caa-8457-4471-bc67-63e5443b86bc
https://buildkite.com/risingwavelabs/pull-request/builds/14723#01855cd3-0f5a-4177-a7ea-de6a1a4669d2

[BugenZhao]

First occurred in #7073, cc @wcy-fdu PTAL.

https://buildkite.com/risingwavelabs/pull-request/builds/14671

[BugenZhao]

BTW, should we run simulation tests on the main branch as well? 🤔 Failures in the PR pipeline are more likely to be ignored and also not reported through the notification.

[lmatz]

+1 and I want to increase the intensity of testing on the main branch because only the PR ones will slow down the speed of development.

[fuyufjh]

It's shocking to me to know some tests only run in PR pipeline 🤯

[fuyufjh]

thread '<unnamed>' panicked at 'mem-table operation conflicts! table_id: 1022, vnode: 23, key: OwnedRow([Some(Int32(3300)), Some(Int64(208157460714926108))]), prev: Insert(Ok(OwnedRow([Some(Int32(3300)), Some(Int64(208157460714926108))]))), new: Insert(Ok(OwnedRow([Some(Int32(3300)), Some(Int64(208157460714926108))])))', /risingwave/src/stream/src/common/table/state_table.rs:505:17

The row (auction: 3300, _row_id: 208157460714926108) from bid source was inserted twice. 🤔

[BugenZhao]

Seems there're duplicated primary keys. Might also be caused by #6836. 🤔

[wcy-fdu]

seems to generate the same row_id

[lmatz]

Again for q7: https://buildkite.com/risingwavelabs/pull-request/builds/14771#01855d6a-1fcb-4909-b94b-7b4fb25d9e2a

[lmatz]

fixed by #7122