Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: impl checkTxAsyncReactor() #168

Merged
merged 5 commits into from
Jan 25, 2021
Merged

feat: impl checkTxAsyncReactor() #168

merged 5 commits into from
Jan 25, 2021

Conversation

jinsan-line
Copy link

@jinsan-line jinsan-line commented Jan 21, 2021

Closes: #167
Related with: #163, https://github.com/line/link/issues/1152

Description

During #163 CI tests, we sometimes failed TestReactorBroadcastTxMessage(). After investigation, I think
we should guarantee that tx, called mempool.CheckTxAsync() first, should call app.CheckTxAsync() first. I think this bug comes from original tendermint. But, in original tendermint, app.CheckTxAsync() is protected by a mutex. Because part of mempool.CheckTxAsync() is serialized by this mutex, it decreases the possibility of failure of the test. Because we removed this mutex, the possibility of failure of the test is increased.

To guarantee the execution order, I introduce a channel and a reactor. It makes the mempool.CheckTxAsync() execute in serial. mempool/reactor.Receive() is executed on its own goroutine. In original tendermint, mempool.CheckTxAsync() could be executed in parallel except app.CheckTx(). Now, mempool.CheckTxAsync() is executed in serial. It looks like decreasing the performance.

Please note that mempool.CheckTxAsync() itself is so cheap but app.CheckTxXXX() is expensive. Now we compare the original and revised. In the original, cheap mempool.CheckTxAsync() itself could be executed in parallel but expensive app.CheckTx() is executed in serial. But, now, cheap mempool.CheckTxAsync() itself is executed in serial but expensive app.CheckTx() could be executed in parallel. So I maintain that the revised is safer and more performant than the original.


For contributor use:

  • Wrote tests
  • Updated CHANGELOG_PENDING.md
  • Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
  • Updated relevant documentation (docs/) and code comments
  • Re-reviewed Files changed in the Github PR explorer

@jinsan-line jinsan-line self-assigned this Jan 21, 2021
@jinsan-line jinsan-line marked this pull request as draft January 21, 2021 03:30
@jinsan-line jinsan-line marked this pull request as ready for review January 21, 2021 12:01
}

// It blocks if we're waiting on Update() or Reap().
func (mem *CListMempool) checkTxAsync(tx types.Tx, txInfo TxInfo, prepareCb func(error), checkTxCb func(*abci.Response)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is prepareCb right? I think it's just an error handler.

Copy link
Author

@jinsan-line jinsan-line Jan 22, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's prepareCheckTx callback. If we follow up the argument for callback name, checkTxCb should ABCIResponse callback.

Copy link

@kukugi kukugi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

Copy link
Contributor

@wetcod wetcod left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jinsan-line jinsan-line merged commit bc7ff22 into Finschia:feat/perf Jan 25, 2021
@jinsan-line jinsan-line deleted the check-tx-async-reactor branch January 25, 2021 02:36
jinsan-line added a commit that referenced this pull request Apr 27, 2021
jinsan-line added a commit that referenced this pull request Apr 28, 2021
* feat: impl checkTxAsyncReactor() (#168)

* fix: tests

* fix: lint errors
egonspace pushed a commit to egonspace/ostracon that referenced this pull request Jul 8, 2021
* feat: impl checkTxAsyncReactor() (Finschia#168)

* fix: tests

* fix: lint errors
egonspace pushed a commit to egonspace/ostracon that referenced this pull request Jul 8, 2021
* feat: impl checkTxAsyncReactor() (Finschia#168)

* fix: tests

* fix: lint errors
egonspace pushed a commit that referenced this pull request Aug 26, 2021
* feat: more prometheus metrics for monitoring performance (#146) (#175)

* chore: config timeout and connection pool (#150) (#171)

* fix: use line/tm-db instead of tendermint/tm-db

* bump up tm-db, iavl; re-apply #201

* chore: use default db backend among the available ones (#212)

* chore: use default db backend among the available ones

* chore: bump up iavl, tm-db

* feat: concurrent checkTx #213; fix tm-db call

* fix: rename TM to OC

* fix: modify key name; tendermint -> ostracon

* chore: rename tendermint to ostracon

* chore: remove mempool.postCheck (#158) (#217)

* fix: error handling after check tx

* fix: typo

* chore: (mempool) remove postCheck and impl reserve

* chore: fix tests

* chore: revise log (remove checkTx.Code)

* chore: add `CONTRACT` for `mem.proxyAppConn.CheckTxAsync()`

* chore: revise numTxs, txsBytes for `ErrMempoolIsFull` in reserve()

* chore: revise to remove redundant `isFull()`

* fix: remove tx from cache when `app errors` or `failed to reserve`

* Revert "chore: revise to remove redundant `isFull()`"

This reverts commit 55990ec.

* fix: revise to call Begin/EndRecheck even though mem.Size() is 0 (#219)

* fix: revise to call Begin/EndRecheck even though `mem.Size()` is 0

* chore: revise local_client.go

* fix: lint error

* chore: recheckTxs() just return if mem.Size() == 0

* feat: concurrent recheckTx (#163) (#221)

* chore: increase the value of maxPerPage (#223)

* chore: fix the type of consensus_block_interval_seconds from histogram to gauge (#224)

* feat: impl checkTxAsyncReactor() (#168) (#225)

* feat: impl checkTxAsyncReactor() (#168)

* fix: tests

* fix: lint errors

* chore: revise abci.Client, Async() interfaces (#169) (#226)

* chore: revise abci.Client, Async() interfaces

* chore: regen mock w/ mockery 2.7.4

* fix: lint error

* fix: test_race

* mempool.Flush() flushes all txs from mempool so it should get `Lock()` instead of `RLock()`

* chore: remove iavl dependency (#228)

* chore: remove iavl dependency

* chore: fix lint error

* fix: add more fixing for abci.Client, Async()

* feat: revise metric for measuring performance

* build: remove needless build tag `!libsecp256k1` (#246)

The build tag makes disable go implementation of secp256k1.
Cause there is no C implementation, a build error will occur when using tag `libsecp256k1`.

* feat: add duration metrics of gauge type (#256)

* perf: optimize checking the txs size (#264)

* perf: optimize checking the txs size

* ci: add GOPRIVATE to workflows

* test: add a unit test

* fix: fix lint errors

* perf: do not flush wal when receive consensus msgs (#273)

* perf: do not flush wal when receive consensus msgs

* fix: ci failure

* fix: lint failure

* fix: ci-e2e build failure

* fix: bump up tm-db

* fix: missing abci api

* fix: bump up tm-db; use memdb

* test: add test case to raise test coverage

* fix: race error

* fix: race error

* fix: race error

* fix: increase e2e test timeout

* fix: add test case for coverage

* fix: e2e docker file

* fix: apply comments

* fix: a Ostracon to an Ostracon
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants