Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lh use conscrypt for jetty #39

Closed
wants to merge 48 commits into from
Closed

Conversation

lhotari
Copy link
Owner

@lhotari lhotari commented May 10, 2021

congbobo184 and others added 30 commits April 29, 2021 15:14
)

## Motivation
`txnIdSortedSet` is redundant, `txnMetaMap` can replace it.
### Motivation

Currently, when the kinesis sink connector fails to send a message, it will not retry. In this case, if `retainOrdering` is enabled, it will lead to subsequent messages can not be sent like the following:
> 17:09:40.923 [crm/messaging-service/messaging-service-reply-0] WARN  org.apache.pulsar.io.kinesis.KinesisSink - Skip acking message to retain ordering with previous failed message prod_extapi.reply.message-Optional[26380226003034]


### Modifications

* Add retry logic for the kinesis sink connector. When sending a message fails, it will retry to send.
…unction logging (apache#10389)

* Completed work for PR apache#7555 to allow customizable function logging

* Fixed license header

* PR apache#10389 Added null check to prevent test failures.

* PR apache#10389 Added debug logging statement

* PR apache#10389 Made changes more robust, added default value, and fixed NullPointerException

* PR apache#10389 Improved backwards compatibility

Co-authored-by: Devin Bost <dbost@overstock.com>
I found some wrong typing in doc when i using golang client. Most of them are package name missioning.
Co-authored-by: k2la <yfuruta@yahoo-corp.jp>
…che#10391)

* PIP-45: Implement load managers locks using coordination service

* Handle executor rejected exception when shutting down

* Fixed SimpleLoadManager still updating the load report on zk client

* Removed unused import

* Fixed imports order

* Simplified listLocks calls

* Addressed comments

* Removed updateOrCreate

* Remove additional put method that's not needed anymore

* Removed unused import

* Fixed LoadBalanceTest

* Fixed ModularLoadManagerImplTest
…onfiguration (apache#10441)

* using ObjectMapper to parse sink config
fixed minor issue in closing some of the PulsarClientImpl resources

* using ObjectMapper to parse source config

* code review changes

* removed unused import
* Improved in max-pending-bytes mechanism for broker

* Fixed imports

* Switched to LongAdder
)

### Motivation
Forcibly deleting the namespace will leave a large number of schemas. 
Topic and namespace are deleted, leaving the schema is meaningless.
If both Topic and namespace are deleted, we cannot find the schemas that need to be deleted based on other info

### Modifications
When the namespace is forcibly deleted, the schema is also deleted
* fixed NPE from pulsar client if we attempt to close the resource when conf.getServiceUrl is blank

* removed redundant check

* catching PulsarClientException instead of Throwable

* catching PulsarClientException instead of Throwable
…che#10467)

* Fix publish callback's entry data is null during ledger rollover

* Don't call recycle() when the callback is null
Co-authored-by: linjunhua <linjh1@jiguang.cn>
…he#10460 (apache#10461)

As reported in apache#10460 (and for other purposes), it's helpful to see the context when a function crashes when trying to open its sink. This PR adds a couple of debug lines to assist in troubleshooting those cases.
)

---

Fixes apache#10332

*Motivation*

Sometimes, the superuser is not only needed the client
role but also needs some data in the authentication data
to check whether the role is the superuser.

The changed interface is introduced from apache#8560,
it's safe to change it since there is no release for that.
)

### Motivation

I temporarily fixed this problem in PR apache#10190.
Now we have found a better way, this way can avoid the seek, then avoid trigger another reconnection.
Thank you @codelipenghui  to troubleshoot this issue with me all night.

We have added a lot of log and found that this issue is caused by some race condition problems. Here is the first reason:
https://github.com/apache/pulsar/blob/f2d72c9fc13a33df584ec1bd96a4c147774b858d/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConsumerImpl.java#L1808-L1818
Now we have an acknowledgmentsGroupingTracker to filter duplicate messages, and this Tracker will be cleaned up after seek.

However, it is possible that the connection is ready and Broker has pushed message, but `acknowledgmentsGroupingTracker.flushAndClean(); ` has not been executed yet. 

Finally hasMessageAvailableAsync returns true, but the message cannot be read because it is filtered by the acknowledgmentsGroupingTracker


### Modifications
clean the tracker when connection was open

### Verifying this change
…#10457)

* PIP-45: Revalidate leader election after session is recovered

* Fixed TopicOwnerTest

* Addresses comments
Fixes apache#10472

### Motivation
GenericJsonRecord should not error out if the non-existent field is queried.

### Modifications

Added a null check and returning null if the field does not exist.

### Verifying this change

Extended one of the tests for GenericJsonRecord.
### Motivation
Topic has been deleted, and the corresponding cache and Listener should also be deleted.
Now only the value of the listeners is deleted, the key is not deleted, and the cache is not deleted  as well.

### Modifications
add clean API to clean cache and listeners

### Verifying this change
…pdated. (apache#10300) (apache#10333)

Fixes apache#10300 

### Motivation

Fix the bug that after updating the partition number of  a partitioned topic, which has topic level auth policy, new producer/consumer of this topic will get error.

### Modifications

In [`org.apache.pulsar.broker.authorization.PulsarAuthorizationProvider#checkPermission`](https://github.com/apache/pulsar/blob/889b9b8e5efc62d2d0cbc761205fba5759c97af0/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authorization/PulsarAuthorizationProvider.java#L394), if current `topicName` is a sub partition topic, also check the permissions of its partitioned topic.
### Motivation
Support listenerName for http lookup

### Modifications
Add listenerName to http API

### Verifying this change
1) test lookup
2) test redirect
* support both STS and default aws creds

* checkstyle

* use provider

* pass creds to properties

* add tests
wolfstudy and others added 18 commits May 6, 2021 16:05
* Update docs for Go client

Signed-off-by: xiaolongran <xiaolongran@tencent.com>

* Fix comments

Signed-off-by: xiaolongran <xiaolongran@tencent.com>

Co-authored-by: xiaolongran <xiaolongran@tencent.com>
Co-authored-by: k2la <yfuruta@yahoo-corp.jp>
* Avoid sending zero permits

* Avoid the same topic name
…che#10493)

* Add instructions to configure code style and checkstyle for IDEA in the README

* Apply suggestions from code review

Co-authored-by: Yu Liu <50226895+Anonymitaet@users.noreply.github.com>

Co-authored-by: Yu Liu <50226895+Anonymitaet@users.noreply.github.com>
…ema and multi versions (apache#10492)

Co-authored-by: Enrico Olivelli <eolivelli@apache.org>
* Add onFilter for Interceptor

* code style
…pache#10446)

Fixes apache#10445

### Motivation

SinkContext should expose Subscription type to the Sink
More context: apache#9927 (comment)

Needed for apache#9927

### Modifications

Added `getSubscriptionType()` to the `SinkContext` interface and `ContextImpl`
…rService.isLeader() (apache#10502)

Fixes apache#10235

### Motivation

According to apache#10235, when `LeaderService` is changing leadership status (like losing leadership, or becoming a leader), the `LeaderService` will be locked with `synchronized` block. Which will block other threads if calling `LeaderService.isLeader()`. This PR changes `ClusterServiceCoordinator` and `WorkerStatsManager` to check if is leader from `MembershipManager`, which will not block other threads if `LeaderService` is at `synchronized` block.

Also, this PR will not resolve the root cause of apache#10235, since there is lack of context about blocked reader for the `FunctionAssignmentTailer`.
…che#10508)

### Motivation

apache#10240 has reverted the changes of the apache#9261 introduced which make the key_shared tests flaky. So it's better to move out the tests from the quarantine group.

### Modifications

Move out the key_shared related tests from the quarantine group.
Fixes a part of apache#5720 

### Motivation

add granularity in topics api authz
### Motivation
There is an `advertisedAddress` in PusarService and an `advertisedAddress` in conf. When `advertisedListener` is set, the values of the two `advertisedAddress` will be inconsistent. 
In the current code, `PusarService.getAdvertisedAddress` is used in some places, and `conf.getAdvertisedAddress` is used in other places.

### Modifications
Put this logic into the config to keep the values in all places consistent.
…er/namespace naming convention. (apache#10486)

Co-authored-by: Chris Bartholomew <c_bartholomew@yahoo.com>
@lhotari lhotari closed this May 10, 2021
@lhotari lhotari deleted the lh-use-conscrypt-for-jetty branch May 10, 2021 14:02
@lhotari lhotari restored the lh-use-conscrypt-for-jetty branch May 10, 2021 14:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Deadlock on Monitoring thread ... LeaderService.isLeader()