Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improve][txn] Allow superusers to abort transactions #19467

Merged

Conversation

nicoloboschi
Copy link
Contributor

Motivation

Super users must be always allowed to abort a transaction even if they're not the original owner.

Modifications

  • Check that only owner or superusers are allowed to perform txn operations (end, add partition and add subscription)

Verifying this change

  • Make sure that the change passes the CI checks.

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

@nicoloboschi nicoloboschi added this to the 3.0.0 milestone Feb 8, 2023
@nicoloboschi nicoloboschi self-assigned this Feb 8, 2023
@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Feb 8, 2023
@congbobo184 congbobo184 merged commit 459a7a5 into apache:master Feb 9, 2023
if (service.isAuthenticationEnabled() && service.isAuthorizationEnabled()) {
return getBrokerService()
.getAuthorizationService()
.isSuperUser(getPrincipal(), getAuthenticationData());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this work correctly for with proxy and original auth data? I will try to review more closely tomorrow.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I've tested it manually. We always use the originalPrincipal - if present - as owner

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One issue with this code is that we should always get this data from the netty event loop that the channel runs on.

@liangyepianzhou liangyepianzhou self-requested a review February 9, 2023 02:52
liangyepianzhou pushed a commit that referenced this pull request Feb 9, 2023
Super users must be always allowed to abort a transaction even if they're not the original owner.

* Check that only owner or superusers are allowed to perform txn operations (end, add partition and add subscription)

(cherry picked from commit 459a7a5)
nicoloboschi added a commit that referenced this pull request Feb 9, 2023
Super users must be always allowed to abort a transaction even if they're not the original owner.

* Check that only owner or superusers are allowed to perform txn operations (end, add partition and add subscription)

(cherry picked from commit 459a7a5)
nicoloboschi added a commit that referenced this pull request Feb 9, 2023
…9467) (#19473)

Co-authored-by: Nicolò Boschi <boschi1997@gmail.com>
return failedFutureTxnNotOwned(txnID);
}
return transactionMetadataStoreService
.addProducedPartitionToTxn(txnID, command.getPartitionsList());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an unsafe call. We must copy the value because command is mutable.

if (service.isAuthenticationEnabled() && service.isAuthorizationEnabled()) {
return getBrokerService()
.getAuthorizationService()
.isSuperUser(checkOwner, getAuthenticationData());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is also unsafe. We should not call getAuthenticationData() from another thread. We can update the thenCompose to thenComposeAsync and run it on the ctx.executor().

return failedFutureTxnTcNotAllowed(txnID);
}
return subscription.endTxn(txnidMostBits, txnidLeastBits, txnAction, lowWaterMark);
}).whenComplete((ignored, e) -> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a nit, but I've been looking closely at the netty implementation, and I noticed that when we write to the channel, netty will schedule an event on the ctx.executor() if the calling thread is not the event loop (the executor). Would it make sense to prevent context switching and just schedule this to run on the ctx's event loop?

@nicoloboschi
Copy link
Contributor Author

@michaeljmarshall I've addressed your comments here: #19517
The last one is very interesting but I think we should do it for all the ServerCnx methods in one shot in another pull request.

nicoloboschi pushed a commit to datastax/pulsar that referenced this pull request Feb 14, 2023
…ache#19467) (apache#19473)

Co-authored-by: Nicolò Boschi <boschi1997@gmail.com>
(cherry picked from commit cb91c4a)
@michaeljmarshall
Copy link
Member

The last one is very interesting but I think we should do it for all the ServerCnx methods in one shot in another pull request.

I completely agree, thanks for addressing my feedback.

michaeljmarshall added a commit that referenced this pull request Apr 6, 2023
### Motivation

This PR builds on #19467. When we modify/abort transactions, we need to make sure that authorization is checked for both the proxy and the client.

### Modifications

* Add a second authorization check when `originalPrincipal` is set in the `ServerCnx`.
* Fix a bug where we were not doing a deep copy of the `SubscriptionsList` object. (Tests caught this bug!)

### Verifying this change

Added a new test to cover some of the changes.

### Does this pull request potentially affect one of the following parts:

This is an internal change.

### Documentation

- [x] `doc-not-needed`

### Matching PR in forked repository

PR in forked repository: michaeljmarshall#38
michaeljmarshall added a commit that referenced this pull request Apr 6, 2023
This PR builds on #19467. When we modify/abort transactions, we need to make sure that authorization is checked for both the proxy and the client.

* Add a second authorization check when `originalPrincipal` is set in the `ServerCnx`.
* Fix a bug where we were not doing a deep copy of the `SubscriptionsList` object. (Tests caught this bug!)

Added a new test to cover some of the changes.

This is an internal change.

- [x] `doc-not-needed`

PR in forked repository: michaeljmarshall#38

(cherry picked from commit f76beda)
michaeljmarshall added a commit that referenced this pull request Apr 7, 2023
This PR builds on #19467. When we modify/abort transactions, we need to make sure that authorization is checked for both the proxy and the client.

* Add a second authorization check when `originalPrincipal` is set in the `ServerCnx`.
* Fix a bug where we were not doing a deep copy of the `SubscriptionsList` object. (Tests caught this bug!)

Added a new test to cover some of the changes.

This is an internal change.

- [x] `doc-not-needed`

PR in forked repository: michaeljmarshall#38

(cherry picked from commit f76beda)
(cherry picked from commit 5a180f78d7636537198a758e1c9416e58d80bf42)
nicoloboschi pushed a commit to datastax/pulsar that referenced this pull request May 11, 2023
This PR builds on apache#19467. When we modify/abort transactions, we need to make sure that authorization is checked for both the proxy and the client.

* Add a second authorization check when `originalPrincipal` is set in the `ServerCnx`.
* Fix a bug where we were not doing a deep copy of the `SubscriptionsList` object. (Tests caught this bug!)

Added a new test to cover some of the changes.

This is an internal change.

- [x] `doc-not-needed`

PR in forked repository: michaeljmarshall#38

(cherry picked from commit f76beda)
(cherry picked from commit 5a180f78d7636537198a758e1c9416e58d80bf42)
(cherry picked from commit 716db37)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants