[Broker] Fix create the dynamic configuration resource if not exist #13420
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nodece
force-pushed
the
create_dynamic_conf
branch
from
f18e814 to
39f84c5
Compare
### Motivation When request the `DELETE: /admin/brokers/configuration/dispatcherMinReadBatchSize`, which return the`org.apache.pulsar.metadata.api.MetadataStoreException$NotFoundException`. The Pulsar does not create the dynamic configuration resource if it does not exist, this is incorrect behavior. Reproduce: ``` $ pulsarctl broker delete-dynamic-config --config dispatcherMinReadBatchSize [✖] code: 500 reason: --- An unexpected error occurred in the server --- Message: Stacktrace: org.apache.pulsar.metadata.api.MetadataStoreException$NotFoundException: at org.apache.pulsar.metadata.cache.impl.MetadataCacheImpl.lambda$readModifyUpdate$8(MetadataCacheImpl.java:181) at java.base/java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1106) at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2235) at org.apache.pulsar.metadata.cache.impl.MetadataCacheImpl.lambda$readModifyUpdate$9(MetadataCacheImpl.java:179) at org.apache.pulsar.metadata.cache.impl.MetadataCacheImpl.executeWithRetry(MetadataCacheImpl.java:300) at org.apache.pulsar.metadata.cache.impl.MetadataCacheImpl.readModifyUpdate(MetadataCacheImpl.java:178) at org.apache.pulsar.broker.resources.BaseResources.setAsync(BaseResources.java:107) at org.apache.pulsar.broker.resources.BaseResources.set(BaseResources.java:97) at org.apache.pulsar.broker.resources.DynamicConfigurationResources.setDynamicConfiguration(DynamicConfigurationResources.java:56) at org.apache.pulsar.broker.admin.impl.BrokersBase.deleteDynamicConfigurationOnZk(BrokersBase.java:417) at org.apache.pulsar.broker.admin.impl.BrokersBase.deleteDynamicConfiguration(BrokersBase.java:171) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$VoidOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:159) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1626) at org.apache.pulsar.broker.web.ResponseHandlerFilter.doFilter(ResponseHandlerFilter.java:67) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:234) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:179) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:388) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:633) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:380) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:386) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Thread.java:829) ``` Affected version: 2.9.x ### Modifications - Add check dynamic configuration resource on `BrokerService.updateDynamicServiceConfiguration()` ### Documentation Need to update docs? - [x] `no-need-doc` Signed-off-by: Zixuan Liu <nodeces@gmail.com>
nodece
force-pushed
the
create_dynamic_conf
branch
from
39f84c5 to
831bbe0
Compare
codelipenghui
requested review from
merlimat,
315157973,
congbobo184,
gaoran10 and
hangc0276
|
@codelipenghui test has been added. |
nodece
force-pushed
the
create_dynamic_conf
branch
2 times, most recently
from
269c73d to
831bbe0
Compare
nodece
force-pushed
the
create_dynamic_conf
branch
from
9c68145 to
6049add
Compare
|
IMHO, We can simply catch the |
|
@Demogorgon314 Pulsar did not create this resource in metadata, so throw the |
Exactly. So we can catch the private synchronized void deleteDynamicConfigurationOnZk(String configName) {
try {
// ...
} catch (RestException re) {
throw re;
} catch (MetadataStoreException.NotFoundException ex) {
LOG.info("xxx");
} catch (Exception ie) {
LOG.error("[{}] Failed to update configuration {}, {}", clientAppId(), configName, ie.getMessage(), ie);
throw new RestException(ie);
}
} |
|
@Demogorgon314 This is a bug. The dynamic configuration resource path never is deleted on metadata, when we delete a dynamic configuration value, we just remove the value stored in the path. |
|
The problem is that should or not be lazy create the config path. Just an idea. I think it is better to create the path when we have dynamic config in there. Of course, the current solution is no problem. |
pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApiBrokerTest.java
Outdated
Show resolved
Hide resolved
...r-common/src/main/java/org/apache/pulsar/broker/resources/DynamicConfigurationResources.java
Outdated
Show resolved
Hide resolved
Signed-off-by: Zixuan Liu <nodeces@gmail.com>
BewareMyPower
approved these changes
Dec 23, 2021
Technoboy-
reviewed
Dec 23, 2021
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/BrokersBase.java
Outdated
Show resolved
Hide resolved
Signed-off-by: Zixuan Liu <nodeces@gmail.com>
codelipenghui
approved these changes
Dec 23, 2021
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/BrokersBase.java
Outdated
Show resolved
Hide resolved
Signed-off-by: Zixuan Liu <nodeces@gmail.com>
Demogorgon314
approved these changes
Dec 23, 2021
Technoboy-
pushed a commit
to Technoboy-/pulsar
that referenced
this pull request
Dec 29, 2021
…pache#13420) ### Motivation When request the `DELETE: /admin/brokers/configuration/dispatcherMinReadBatchSize`, which return the`org.apache.pulsar.metadata.api.MetadataStoreException$NotFoundException`. The Pulsar does not create the dynamic configuration resource path on metadata if it does not exist, this behavior is different with Pulsar 2.8.
codelipenghui
pushed a commit
that referenced
this pull request
Dec 30, 2021
…13420) ### Motivation When request the `DELETE: /admin/brokers/configuration/dispatcherMinReadBatchSize`, which return the`org.apache.pulsar.metadata.api.MetadataStoreException$NotFoundException`. The Pulsar does not create the dynamic configuration resource path on metadata if it does not exist, this behavior is different with Pulsar 2.8. (cherry picked from commit 3a1e8da)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
When request the
DELETE: /admin/brokers/configuration/dispatcherMinReadBatchSize, which return theorg.apache.pulsar.metadata.api.MetadataStoreException$NotFoundException. The Pulsar does not create the dynamic configuration resource path on metadata if it does not exist, this behavior is different with Pulsar 2.8.Reproduce:
Affected version: 2.9.x
Modifications
BrokerService.updateDynamicServiceConfiguration()Documentation
Need to update docs?
no-need-docSigned-off-by: Zixuan Liu nodeces@gmail.com