Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Transaction] Fix transaction sequenceId generate error. #13209

Conversation

congbobo184
Copy link
Contributor

Motivation

now, we recover transaction sequenceId from lastConfirmEntry or managedLedger properties. But update managedLedger properties is not a Synchronize op, so it will recover error sequenceId.

Modifications

use ManagedLedgerInterceptor will fix this problem.

  1. use interceptor.onUpdateManagedLedgerInfo, when ledger roll over, it will update current sequenceId to managedLedger properties.
  2. use interceptor.onManagedLedgerPropertiesInitialize, first we will recover sequenceId from managedLedger properties.
  3. use interceptor.onManagedLedgerLastLedgerInitialize, when tc recover and has a effective lastConfirmEntry, we will change the sequenceID after interceptor.onManagedLedgerPropertiesInitialize
  4. If a new TC, it can't recover from interceptor.onManagedLedgerPropertiesInitialize and interceptor.onManagedLedgerLastLedgerInitialize, it will use the initial sequenceId -1

Verifying this change

Add the tests for it

Does this pull request potentially affect one of the following parts:
If yes was chosen, please highlight the changes

Dependencies (does it add or upgrade a dependency): (no)
The public API: (no)
The schema: (no)
The default values of configurations: (no)
The wire protocol: (no)
The rest endpoints: (no)
The admin cli options: (no)
Anything that affects deployment: (no)

@github-actions
Copy link

github-actions bot commented Dec 9, 2021

@congbobo184:Thanks for your contribution. For this PR, do we need to update docs?
(The PR template contains info about doc, which helps others know more about the changes. Can you provide doc-related info in this and future PR descriptions? Thanks)

@congbobo184 congbobo184 added the doc-not-needed Your PR changes do not impact docs label Dec 9, 2021
@github-actions
Copy link

github-actions bot commented Dec 9, 2021

@congbobo184:Thanks for providing doc info!

Copy link
Contributor

@eolivelli eolivelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this change compatible with existing Pulsar clusters ?

@congbobo184
Copy link
Contributor Author

is this change compatible with existing Pulsar clusters ?

It will be compatible

@codelipenghui codelipenghui added this to the 2.10.0 milestone Dec 10, 2021
Copy link
Contributor

@eolivelli eolivelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@eolivelli eolivelli merged commit 0994254 into apache:master Dec 10, 2021
LedgerEntry ledgerEntry = entries.getEntry(lh.getLastAddConfirmed());
if (ledgerEntry != null) {
TransactionMetadataEntry lastConfirmEntry = new TransactionMetadataEntry();
ByteBuf buffer = ledgerEntry.getEntryBuffer();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The buffer should be released after getting the max local txn ID.

} else {
if (entries != null) {
try {
LedgerEntry ledgerEntry = entries.getEntry(lh.getLastAddConfirmed());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TheledgerEntryshould be closed after getting the max local txn ID.


private volatile long maxLocalTxnId = -1;
@Getter
private final AtomicLong sequenceId = new AtomicLong(TC_ID_NOT_USED);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'd better return long not AtomicLong for the getter method.

return txnLog.initialize().thenApply(__ ->
new MLTransactionMetadataStore(transactionCoordinatorId, txnLog, timeoutTracker, recoverTracker));
new MLTransactionMetadataStore(transactionCoordinatorId, txnLog, timeoutTracker,
recoverTracker, mlTransactionLogInterceptor.getSequenceId()));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's better to introduce modify sequence ID method for mlTransactionLogInterceptor, share AtomicLong across multiple instances is not good for maintainance

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/transaction cherry-picked/branch-2.9 Archived: 2.9 is end of life doc-not-needed Your PR changes do not impact docs release/2.9.2
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants