Support txn management in db level isolation and use ArrayDeque to improve txn task performance #3369
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
morningman
reviewed
Apr 24, 2020
fe/src/main/java/org/apache/doris/transaction/DatabaseTransactionMgr.java
Show resolved
Hide resolved
fe/src/main/java/org/apache/doris/transaction/DatabaseTransactionMgr.java
Show resolved
Hide resolved
fe/src/main/java/org/apache/doris/transaction/GlobalTransactionMgr.java
Outdated
Show resolved
Hide resolved
fe/src/main/java/org/apache/doris/transaction/GlobalTransactionMgr.java
Outdated
Show resolved
Hide resolved
morningman
requested changes
May 5, 2020
| } | ||
|
|
||
| Catalog.getCurrentGlobalTransactionMgr().abortTransaction(request.getTxnId(), | ||
| long dbId = Catalog.getInstance().getDb(request.getDb()).getId(); |
Contributor
There was a problem hiding this comment.
getDb() may return null if database does not exist yet.
| public DatabaseTransactionMgr getDatabaseTransactioMgr(long dbId) { | ||
| DatabaseTransactionMgr dbTransactionMgr = dbIdToDatabaseTransactionMgrs.get(dbId); | ||
| if (dbTransactionMgr == null) { | ||
| throw new NullPointerException("databaseTransactionMgr[" + dbId + "] does not exist"); |
Contributor
There was a problem hiding this comment.
Not using NullPointerException, you can use TransactionException
| } | ||
|
|
||
| public void addDatabaseTransactionMgr(Long dbId, EditLog editLog) { | ||
| dbIdToDatabaseTransactionMgrs.put(dbId, new DatabaseTransactionMgr(dbId, editLog)); |
Contributor
There was a problem hiding this comment.
Is it more safe to use putIfAbsent() method?
| } | ||
|
|
||
| checkRunningTxnExceedLimit(dbId, sourceType); | ||
| checkRunningTxnExceedLimit(dbTransactionMgr, sourceType); |
Contributor
There was a problem hiding this comment.
I think checkRunningTxnExceedLimit() this method can be moved into the DbTransactionMgr
fe/src/main/java/org/apache/doris/transaction/GlobalTransactionMgr.java
Outdated
Show resolved
Hide resolved
Contributor
|
@caiconghui I have some comments left. And could you please resolve the conflicts? |
caiconghui
force-pushed
the
db_txn
branch
2 times, most recently
from
c6fb244 to
4f38eb0
Compare
morningman
added
area/load
morningman
requested changes
May 7, 2020
fe/src/main/java/org/apache/doris/transaction/GlobalTransactionMgr.java
Outdated
Show resolved
Hide resolved
| public DatabaseTransactionMgr getDatabaseTransactioMgr(long dbId) throws AnalysisException { | ||
| DatabaseTransactionMgr dbTransactionMgr = dbIdToDatabaseTransactionMgrs.get(dbId); | ||
| if (dbTransactionMgr == null) { | ||
| throw new AnalysisException("databaseTransactionMgr[" + dbId + "] does not exist"); |
Contributor
There was a problem hiding this comment.
AnalysisException is not suitable here.
But it can be modified next time
fe/src/main/java/org/apache/doris/transaction/GlobalTransactionMgr.java
Outdated
Show resolved
Hide resolved
fe/src/main/java/org/apache/doris/transaction/GlobalTransactionMgr.java
Outdated
Show resolved
Hide resolved
fe/src/main/java/org/apache/doris/transaction/GlobalTransactionMgr.java
Outdated
Show resolved
Hide resolved
fe/src/main/java/org/apache/doris/transaction/GlobalTransactionMgr.java
Outdated
Show resolved
Hide resolved
| } | ||
| } | ||
| return txnIds; | ||
| return txnInfos.size() > limit ? new ArrayList<>(txnInfos.subList(0, limit)) : txnInfos; |
Contributor
There was a problem hiding this comment.
Suggested change
| return txnInfos.size() > limit ? new ArrayList<>(txnInfos.subList(0, limit)) : txnInfos; | |
| return txnInfos.size() > limit ? txnInfos.subList(0, limit) : txnInfos; |
Contributor
Author
There was a problem hiding this comment.
@morningman subList is a view of origin list, so is it a good way to directly return sublist?
Contributor
There was a problem hiding this comment.
I think it's OK, cause no one will modify that list.
fe/src/main/java/org/apache/doris/transaction/GlobalTransactionMgr.java
Outdated
Show resolved
Hide resolved
This was referenced May 11, 2020
Closed
acelyc111
pushed a commit
to acelyc111/incubator-doris
that referenced
this pull request
Jan 20, 2021
…rove txn task performance (apache#3369) This PR is the first step to make Doris stream load more robust with higher concurrent performance(apache#3368)，the main work is to support txn management in db level isolation and use ArrayDeque to stored final status txns.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR is the first step to make doris stream load more robust with higher concurrent performance(#3368)，the main work is to support txn management in db level isolation，and use ArrayDeque to stored final status txns