Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] Transient error occurred: The link '...' is force detached #34588

Closed
3 tasks done
AndreasPetersen opened this issue Apr 21, 2023 · 6 comments
Closed
3 tasks done

[BUG] Transient error occurred: The link '...' is force detached #34588

AndreasPetersen opened this issue Apr 21, 2023 · 6 comments
Assignees
@anuchandy
Labels
Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. needs-team-triage Workflow: This issue needs the team to triage. question The issue doesn't require a change to the product in order to be resolved. Most issues start as that Service Bus

Comments

@AndreasPetersen
Copy link

AndreasPetersen commented Apr 21, 2023 •
edited
Loading

Describe the bug
After running our service since 2023-04-19 around 15:00, on 2023-04-20 at around 12:40 our services stops receiving messages. There are no warning or error messages saying why. At 16:57 the services starts receiving the errors below. This continues until at least 2023-04-21 10:00 where I'm writing this bug report. Meanwhile no new messages are received.

We are running two pods on a Kubernetes cluster, and the error occurred on both pods at the same time.

I can see that there are related issues reported, but these are closed and locked: #18890, #32919

Exception or Stack Trace
Specifically the warning/error logs. The rest are included for context.

The errors occurs every 10 minutes.

2023-04-21 10:15:05,139 INFO [com.azu.cor.amq.imp.han.ReceiveLinkHandler] (reactor-executor-1) {} {"az.sdk.message":"onLinkRemoteClose","connectionId":"MF_7c2060_1681868985635","errorCondition":"amqp:link:detach-forced","errorDescription":"The link 'G0S1:100206360:generate_history_579df9_1681868985839' is force detached. Code: consumer(link2731). Details: AmqpMessageConsumer.IdleTimerExpired: Idle timeout: 00:10:00. TrackingId:a0c677c10000070c00000aab644243b1_G0S1_B7S1, SystemTracker:G0S1:100206360:generate_history_579df9_1681868985839, Timestamp:2023-04-21T08:15:05","linkName":"generate_history_579df9_1681868985839","entityPath":"generate_history"}
2023-04-21 10:15:05,139 INFO [com.azu.cor.amq.imp.han.ReceiveLinkHandler] (reactor-executor-1) {} {"az.sdk.message":"Local link state is not closed.","connectionId":"MF_7c2060_1681868985635","linkName":"generate_history_579df9_1681868985839","entityPath":"generate_history","state":"ACTIVE"}
2023-04-21 10:15:05,139 INFO [com.azu.cor.amq.imp.ReactorReceiver] (reactor-executor-1) {} {"az.sdk.message":"Freeing resources due to error.","connectionId":"MF_7c2060_1681868985635","entityPath":"generate_history","linkName":"generate_history_579df9_1681868985839"}
2023-04-21 10:15:05,139 WARN [com.azu.mes.ser.imp.ServiceBusReceiveLinkProcessor] (reactor-executor-1) {} {"az.sdk.message":"Transient error occurred.","exception":"The link 'G0S1:100206360:generate_history_579df9_1681868985839' is force detached. Code: consumer(link2731). Details: AmqpMessageConsumer.IdleTimerExpired: Idle timeout: 00:10:00. TrackingId:a0c677c10000070c00000aab644243b1_G0S1_B7S1, SystemTracker:G0S1:100206360:generate_history_579df9_1681868985839, Timestamp:2023-04-21T08:15:05, errorContext[NAMESPACE: myservicebus.servicebus.windows.net. ERROR CONTEXT: N/A, PATH: generate_history, REFERENCE_ID: generate_history_579df9_1681868985839, LINK_CREDIT: 0]","linkName":"n/a","entityPath":"n/a","attempt":1,"retryAfter":4511}
2023-04-21 10:15:05,139 INFO [com.azu.cor.amq.imp.han.ReceiveLinkHandler] (reactor-executor-1) {} {"az.sdk.message":"onLinkFinal","connectionId":"MF_7c2060_1681868985635","linkName":"generate_history_579df9_1681868985839","entityPath":"generate_history"}
2023-04-21 10:15:09,651 INFO [com.azu.mes.ser.imp.ServiceBusReceiveLinkProcessor] (parallel-1) {} Requesting a new AmqpReceiveLink from upstream.
2023-04-21 10:15:09,651 INFO [com.azu.mes.ser.imp.ServiceBusReceiveLinkProcessor] (parallel-1) {} {"az.sdk.message":"Setting next AMQP receive link.","linkName":"generate_history_579df9_1681868985839","entityPath":"generate_history"}
2023-04-21 10:15:09,651 INFO [com.azu.mes.ser.imp.ServiceBusReceiveLinkProcessor] (parallel-1) {} Not adding credits. No downstream subscribers or items requested.
2023-04-21 10:15:09,651 WARN [com.azu.mes.ser.imp.ServiceBusReceiveLinkProcessor] (boundedElastic-28) {} {"az.sdk.message":"Transient error occurred.","exception":"The link 'G0S1:100206360:generate_history_579df9_1681868985839' is force detached. Code: consumer(link2731). Details: AmqpMessageConsumer.IdleTimerExpired: Idle timeout: 00:10:00. TrackingId:a0c677c10000070c00000aab644243b1_G0S1_B7S1, SystemTracker:G0S1:100206360:generate_history_579df9_1681868985839, Timestamp:2023-04-21T08:15:05, errorContext[NAMESPACE: myservicebus.servicebus.windows.net. ERROR CONTEXT: N/A, PATH: generate_history, REFERENCE_ID: generate_history_579df9_1681868985839, LINK_CREDIT: 0]","linkName":"n/a","entityPath":"n/a","attempt":1,"retryAfter":4511}
2023-04-21 10:15:09,652 INFO [com.mic.aad.msa.AcquireTokenSilentSupplier] (Thread-380) {} Returning token from cache
2023-04-21 10:15:09,653 INFO [com.azu.ide.ClientSecretCredential] (Thread-380) {} Azure Identity => getToken() result for scopes [https://servicebus.azure.net/.default]: SUCCESS
2023-04-21 10:15:09,865 INFO [com.azu.cor.amq.imp.ActiveClientTokenManager] (reactor-executor-1) {} {"az.sdk.message":"Scheduling refresh token task.","scopes":"https://servicebus.azure.net/.default"}
2023-04-21 10:15:09,865 INFO [com.azu.cor.amq.imp.ReactorSession] (reactor-executor-1) {} {"az.sdk.message":"Creating a new receiver link.","connectionId":"MF_7c2060_1681868985635","sessionName":"generate_history","linkName":"generate_history_579df9_1681868985839"}
2023-04-21 10:15:09,899 INFO [com.azu.cor.amq.imp.han.ReceiveLinkHandler] (reactor-executor-1) {} {"az.sdk.message":"onLinkRemoteOpen","connectionId":"MF_7c2060_1681868985635","entityPath":"generate_history","linkName":"generate_history_579df9_1681868985839","remoteSource":"Source{address='generate_history', durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false, dynamicNodeProperties=null, distributionMode=null, filter=null, defaultOutcome=null, outcomes=null, capabilities=null}"}
2023-04-21 10:15:14,163 INFO [com.azu.mes.ser.imp.ServiceBusReceiveLinkProcessor] (parallel-1) {} Requesting a new AmqpReceiveLink from upstream.
2023-04-21 10:15:14,163 INFO [com.azu.mes.ser.imp.ServiceBusReceiveLinkProcessor] (parallel-1) {} {"az.sdk.message":"Setting next AMQP receive link.","linkName":"generate_history_579df9_1681868985839","entityPath":"generate_history"}
2023-04-21 10:15:14,163 INFO [com.azu.mes.ser.imp.ServiceBusReceiveLinkProcessor] (parallel-1) {} Not adding credits. No downstream subscribers or items requested.
2023-04-21 10:15:14,164 INFO [com.azu.cor.amq.imp.ReactorSession] (parallel-1) {} {"az.sdk.message":"Returning existing receive link.","connectionId":"MF_7c2060_1681868985635","linkName":"generate_history_579df9_1681868985839","entityPath":"generate_history"}

To Reproduce
Steps to reproduce the behavior:

Unknown

Code Snippet

public void Flux<ServiceBusReceivedMessage> receiveMessages(Config config) {
    ServiceBusReceiverAsyncClient serviceBusReceiverAsyncClient = getServiceBusReceiverAsyncClient(config);
    return serviceBusReceiverAsyncClient.receiveMessages();
}

private ServiceBusReceiverAsyncClient getServiceBusReceiverAsyncClient(Config config) {
    return createServiceBusClientBuilder(config)
            .receiver()
            .queueName(config.getValue("queue-name", String.class))
            .disableAutoComplete()
            .buildAsyncClient();
}

private ServiceBusClientBuilder createServiceBusClientBuilder(Config config) {
    TokenCredential credential = getTokenCredential(config);
    String namespace = config.getValue("namespace", String.class);

    ServiceBusClientBuilder serviceBusClientBuilder = new ServiceBusClientBuilder()
            .transportType(AmqpTransportType.AMQP_WEB_SOCKETS)
            .credential(namespace, credential)
            .proxyOptions(getAmqpProxyOptions(config));
    return serviceBusClientBuilder;
}

private ProxyOptions getAmqpProxyOptions(Config config) {
    Proxy proxy = getAmqpProxy(config);
    return new ProxyOptions(ProxyAuthenticationType.NONE, proxy, null, null);
}

private Proxy getAmqpProxy(Config config) {
    if (config.getOptionalValue("use-proxy-for-amqp-connection", Boolean.class).orElse(true)) {
        return new Proxy(Proxy.Type.HTTP, getInetSocketAddress(config));
    } else {
        return Proxy.NO_PROXY;
    }
}

private TokenCredential getTokenCredential(Config config) {
    return new ClientSecretCredentialBuilder()
            .tenantId(config.getValue("tenant-id", String.class))
            .clientId(config.getValue("client-id", String.class))
            .clientSecret(config.getValue("client-secret", String.class))
            .httpClient(getHttpClientForLogin(config))
            .build();
}

private HttpClient getHttpClientForLogin(Config config) {
    return new NettyAsyncHttpClientBuilder()
            .proxy(getHttpProxyOptions(config))
            .build();
}

private com.azure.core.http.ProxyOptions getHttpProxyOptions(Config config) {
    return new com.azure.core.http.ProxyOptions(com.azure.core.http.ProxyOptions.Type.HTTP, getInetSocketAddress(config));
}

private static InetSocketAddress getInetSocketAddress(Config config) {
    String proxyHost = config.getOptionalValue("proxy-host", String.class)
            .orElse("my-coperate-proxy");
    int proxyPort = config.getOptionalValue("proxy-port", Integer.class)
            .orElse(8080);
    return new InetSocketAddress(proxyHost, proxyPort);
}

Expected behavior
Being able to re-establish connection and continue receiving messages.

Setup (please complete the following information):

  • OS: registry.access.redhat.com/ubi8/openjdk-17
  • Library/Libraries: com.azure:azure-messaging-servicebus:7.13.0 and com.azure:azure-identity:1.5.4
  • Java version: 17
  • App Server/Environment: Kubernetes v1.24.11+af0420d
  • Frameworks: Quarkus 2.16.4

Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

  • Bug Description Added
  • Repro Steps Added
  • Setup information Added
@github-actions github-actions bot added Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. needs-team-triage Workflow: This issue needs the team to triage. question The issue doesn't require a change to the product in order to be resolved. Most issues start as that Service Bus labels Apr 21, 2023
@azure-sdk azure-sdk added the needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. label Apr 21, 2023
@JimSuplizio JimSuplizio removed the needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. label Apr 21, 2023
@AndreasPetersen
Copy link
Author

We restarted our pods, and they started getting the error after about 26 minutes. In the mean time no messages were received. It's not immediately obvious from the logs why no messages were received during those 26 minutes:

2023-04-21 14:17:34,827 INFO  [com.azu.cor.amq.imp.ReactorSession] (parallel-1) {} {"az.sdk.message":"Returning existing receive link.","connectionId":"MF_dae54d_1682077900713","linkName":"generate_history_359c54_1682077900924","entityPath":"generate_history"}
2023-04-21 14:17:34,827 INFO  [com.azu.mes.ser.imp.ServiceBusReceiveLinkProcessor] (parallel-1) {} Not adding credits. No downstream subscribers or items requested.
2023-04-21 14:17:34,826 INFO  [com.azu.mes.ser.imp.ServiceBusReceiveLinkProcessor] (parallel-1) {} {"az.sdk.message":"Setting next AMQP receive link.","linkName":"generate_history_359c54_1682077900924","entityPath":"generate_history"}
2023-04-21 14:17:34,826 INFO  [com.azu.mes.ser.imp.ServiceBusReceiveLinkProcessor] (parallel-1) {} Requesting a new AmqpReceiveLink from upstream.
2023-04-21 14:17:30,509 INFO  [com.azu.cor.amq.imp.han.ReceiveLinkHandler] (reactor-executor-1) {} {"az.sdk.message":"onLinkRemoteOpen","connectionId":"MF_dae54d_1682077900713","entityPath":"generate_history","linkName":"generate_history_359c54_1682077900924","remoteSource":"Source{address='generate_history', durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false, dynamicNodeProperties=null, distributionMode=null, filter=null, defaultOutcome=null, outcomes=null, capabilities=null}"}
2023-04-21 14:17:30,476 INFO  [com.azu.cor.amq.imp.ReactorSession] (reactor-executor-1) {} {"az.sdk.message":"Creating a new receiver link.","connectionId":"MF_dae54d_1682077900713","sessionName":"generate_history","linkName":"generate_history_359c54_1682077900924"}
2023-04-21 14:17:30,476 INFO  [com.azu.cor.amq.imp.ActiveClientTokenManager] (reactor-executor-1) {} {"az.sdk.message":"Scheduling refresh token task.","scopes":"https://servicebus.azure.net/.default"}
2023-04-21 14:17:30,317 INFO  [com.azu.ide.ClientSecretCredential] (Thread-13) {} Azure Identity => getToken() result for scopes [https://servicebus.azure.net/.default]: SUCCESS
2023-04-21 14:17:30,316 INFO  [com.mic.aad.msa.AcquireTokenSilentSupplier] (Thread-13) {} Returning token from cache
2023-04-21 14:17:30,314 WARN  [com.azu.mes.ser.imp.ServiceBusReceiveLinkProcessor] (boundedElastic-2) {} {"az.sdk.message":"Transient error occurred.","exception":"The link 'G4S2:104094190:generate_history_359c54_1682077900924' is force detached. Code: consumer(link2783). Details: AmqpMessageConsumer.IdleTimerExpired: Idle timeout: 00:10:00. TrackingId:a0c677c10000070c00000adf644278d1_G4S2_B7S1, SystemTracker:G4S2:104094190:generate_history_359c54_1682077900924, Timestamp:2023-04-21T12:17:25, errorContext[NAMESPACE: myservicebus.servicebus.windows.net. ERROR CONTEXT: N/A, PATH: generate_history, REFERENCE_ID: generate_history_359c54_1682077900924, LINK_CREDIT: 0]","linkName":"n/a","entityPath":"n/a","attempt":1,"retryAfter":4511}
2023-04-21 14:17:30,313 INFO  [com.azu.mes.ser.imp.ServiceBusReceiveLinkProcessor] (parallel-1) {} Not adding credits. No downstream subscribers or items requested.
2023-04-21 14:17:30,312 INFO  [com.azu.mes.ser.imp.ServiceBusReceiveLinkProcessor] (parallel-1) {} {"az.sdk.message":"Setting next AMQP receive link.","linkName":"generate_history_359c54_1682077900924","entityPath":"generate_history"}
2023-04-21 14:17:30,312 INFO  [com.azu.mes.ser.imp.ServiceBusReceiveLinkProcessor] (parallel-1) {} Requesting a new AmqpReceiveLink from upstream.
2023-04-21 14:17:25,804 INFO  [com.azu.cor.amq.imp.han.ReceiveLinkHandler] (reactor-executor-1) {} {"az.sdk.message":"onLinkFinal","connectionId":"MF_dae54d_1682077900713","linkName":"generate_history_359c54_1682077900924","entityPath":"generate_history"}
2023-04-21 14:17:25,800 WARN  [com.azu.mes.ser.imp.ServiceBusReceiveLinkProcessor] (reactor-executor-1) {} {"az.sdk.message":"Transient error occurred.","exception":"The link 'G4S2:104094190:generate_history_359c54_1682077900924' is force detached. Code: consumer(link2783). Details: AmqpMessageConsumer.IdleTimerExpired: Idle timeout: 00:10:00. TrackingId:a0c677c10000070c00000adf644278d1_G4S2_B7S1, SystemTracker:G4S2:104094190:generate_history_359c54_1682077900924, Timestamp:2023-04-21T12:17:25, errorContext[NAMESPACE: myservicebus.servicebus.windows.net. ERROR CONTEXT: N/A, PATH: generate_history, REFERENCE_ID: generate_history_359c54_1682077900924, LINK_CREDIT: 0]","linkName":"n/a","entityPath":"n/a","attempt":1,"retryAfter":4511}
2023-04-21 14:17:25,797 INFO  [com.azu.cor.amq.imp.ReactorReceiver] (reactor-executor-1) {} {"az.sdk.message":"Freeing resources due to error.","connectionId":"MF_dae54d_1682077900713","entityPath":"generate_history","linkName":"generate_history_359c54_1682077900924"}
2023-04-21 14:17:25,795 INFO  [com.azu.cor.amq.imp.han.ReceiveLinkHandler] (reactor-executor-1) {} {"az.sdk.message":"Local link state is not closed.","connectionId":"MF_dae54d_1682077900713","linkName":"generate_history_359c54_1682077900924","entityPath":"generate_history","state":"ACTIVE"}
2023-04-21 14:17:25,794 INFO  [com.azu.cor.amq.imp.han.ReceiveLinkHandler] (reactor-executor-1) {} {"az.sdk.message":"onLinkRemoteClose","connectionId":"MF_dae54d_1682077900713","errorCondition":"amqp:link:detach-forced","errorDescription":"The link 'G4S2:104094190:generate_history_359c54_1682077900924' is force detached. Code: consumer(link2783). Details: AmqpMessageConsumer.IdleTimerExpired: Idle timeout: 00:10:00. TrackingId:a0c677c10000070c00000adf644278d1_G4S2_B7S1, SystemTracker:G4S2:104094190:generate_history_359c54_1682077900924, Timestamp:2023-04-21T12:17:25","linkName":"generate_history_359c54_1682077900924","entityPath":"generate_history"}
2023-04-21 13:57:11,643 INFO  [com.azu.cor.amq.imp.ActiveClientTokenManager] (reactor-executor-1) {} {"az.sdk.message":"Scheduling refresh token task.","scopes":"https://servicebus.azure.net/.default"}
2023-04-21 13:57:11,610 INFO  [com.azu.cor.amq.imp.han.ReceiveLinkHandler] (reactor-executor-1) {} {"az.sdk.message":"onLinkRemoteOpen","connectionId":"MF_dae54d_1682077900713","entityPath":"generate_history/$management","linkName":"generate_history-mgmt:receiver","remoteSource":"Source{address='generate_history/$management', durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false, dynamicNodeProperties=null, distributionMode=null, filter=null, defaultOutcome=null, outcomes=null, capabilities=null}"}
2023-04-21 13:57:11,610 INFO  [com.azu.cor.amq.imp.AmqpChannelProcessor] (reactor-executor-1) {} {"az.sdk.message":"Channel is now active.","connectionId":"MF_dae54d_1682077900713","entityPath":"generate_history/$management"}
2023-04-21 13:57:11,609 INFO  [com.azu.cor.amq.imp.han.SendLinkHandler] (reactor-executor-1) {} {"az.sdk.message":"onLinkRemoteOpen","connectionId":"MF_dae54d_1682077900713","linkName":"generate_history-mgmt:sender","entityPath":"generate_history/$management","remoteTarget":"Target{address='generate_history/$management', durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false, dynamicNodeProperties=null, capabilities=null}"}
2023-04-21 13:57:11,557 INFO  [com.azu.cor.amq.imp.AmqpChannelProcessor] (reactor-executor-1) {} {"az.sdk.message":"Setting next AMQP channel.","connectionId":"MF_dae54d_1682077900713","entityPath":"generate_history/$management"}
2023-04-21 13:57:11,557 INFO  [com.azu.cor.amq.imp.ReactorConnection] (reactor-executor-1) {} {"az.sdk.message":"Emitting new response channel.","connectionId":"MF_dae54d_1682077900713","entityPath":"generate_history/$management","linkName":"generate_history-mgmt"}
2023-04-21 13:57:11,556 INFO  [com.azu.cor.amq.imp.han.SessionHandler] (reactor-executor-1) {} {"az.sdk.message":"onSessionRemoteOpen","connectionId":"MF_dae54d_1682077900713","sessionName":"generate_history-mgmt-session","sessionIncCapacity":0,"sessionOutgoingWindow":2147483647}
2023-04-21 13:57:11,526 INFO  [com.azu.ide.ClientSecretCredential] (Thread-12) {} Azure Identity => getToken() result for scopes [https://servicebus.azure.net/.default]: SUCCESS
2023-04-21 13:57:11,525 INFO  [com.mic.aad.msa.AcquireTokenSilentSupplier] (Thread-12) {} Returning token from cache
2023-04-21 13:57:11,521 INFO  [com.azu.mes.ser.imp.ServiceBusReactorAmqpConnection] (parallel-1) {} {"az.sdk.message":"Creating management node.","linkName":"generate_history-mgmt","entityPath":"generate_history","address":"generate_history/$management"}
2023-04-21 13:56:21,620 INFO  [io.sma.rea.mes.pro.con.WorkerPoolRegistry] (vert.x-eventloop-thread-1) {parentId=f4809c95849bfe18, sampled=true, spanId=7a75c206b98c9acb, traceId=c57ab1e64e936ae2b893a8cff3ec20e6} Created worker pool named in-history with concurrency of 20
2023-04-21 13:51:45,291 INFO  [com.azu.cor.amq.imp.han.ReceiveLinkHandler] (reactor-executor-1) {} {"az.sdk.message":"onLinkRemoteOpen","connectionId":"MF_dae54d_1682077900713","entityPath":"generate_history","linkName":"generate_history_359c54_1682077900924","remoteSource":"Source{address='generate_history', durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false, dynamicNodeProperties=null, distributionMode=null, filter=null, defaultOutcome=null, outcomes=null, capabilities=null}"}
2023-04-21 13:51:45,238 INFO  [com.azu.cor.amq.imp.ReactorSession] (reactor-executor-1) {} {"az.sdk.message":"Returning existing receive link.","connectionId":"MF_dae54d_1682077900713","linkName":"generate_history_359c54_1682077900924","entityPath":"generate_history"}
2023-04-21 13:51:45,220 INFO  [com.azu.mes.ser.imp.ServiceBusReceiveLinkProcessor] (reactor-executor-1) {} {"az.sdk.message":"Setting next AMQP receive link.","linkName":"generate_history_359c54_1682077900924","entityPath":"generate_history"}
2023-04-21 13:51:45,210 INFO  [com.azu.cor.amq.imp.ReactorSession] (reactor-executor-1) {} {"az.sdk.message":"Creating a new receiver link.","connectionId":"MF_dae54d_1682077900713","sessionName":"generate_history","linkName":"generate_history_359c54_1682077900924"}
2023-04-21 13:51:45,204 INFO  [com.azu.cor.amq.imp.ActiveClientTokenManager] (reactor-executor-1) {} {"az.sdk.message":"Scheduling refresh token task.","scopes":"https://servicebus.azure.net/.default"}
2023-04-21 13:51:45,029 INFO  [com.azu.ide.ClientSecretCredential] (Thread-10) {} Azure Identity => getToken() result for scopes [https://servicebus.azure.net/.default]: SUCCESS
2023-04-21 13:51:43,315 INFO  [com.azu.cor.amq.imp.han.ReceiveLinkHandler] (reactor-executor-1) {} {"az.sdk.message":"onLinkRemoteOpen","connectionId":"MF_dae54d_1682077900713","entityPath":"$cbs","linkName":"cbs:receiver","remoteSource":"Source{address='$cbs', durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false, dynamicNodeProperties=null, distributionMode=null, filter=null, defaultOutcome=null, outcomes=null, capabilities=null}"}
2023-04-21 13:51:43,315 INFO  [com.azu.cor.amq.imp.AmqpChannelProcessor] (reactor-executor-1) {} {"az.sdk.message":"Channel is now active.","connectionId":"MF_dae54d_1682077900713","entityPath":"$cbs"}
2023-04-21 13:51:43,312 INFO  [com.azu.cor.amq.imp.han.SendLinkHandler] (reactor-executor-1) {} {"az.sdk.message":"onLinkRemoteOpen","connectionId":"MF_dae54d_1682077900713","linkName":"cbs:sender","entityPath":"$cbs","remoteTarget":"Target{address='$cbs', durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false, dynamicNodeProperties=null, capabilities=null}"}
2023-04-21 13:51:43,221 INFO  [com.azu.cor.amq.imp.AmqpChannelProcessor] (reactor-executor-1) {} {"az.sdk.message":"Next AMQP channel received.","connectionId":"MF_dae54d_1682077900713","entityPath":"$cbs","subscriberId":"un_dab5d5_1682077903038"}
2023-04-21 13:51:43,138 INFO  [com.azu.cor.uti.tra.DefaultTracerProvider] (reactor-executor-1) {} Found Tracer implementation on the classpath: com.azure.core.tracing.opentelemetry.OpenTelemetryTracer
2023-04-21 13:51:43,076 INFO  [com.azu.cor.amq.imp.AmqpChannelProcessor] (reactor-executor-1) {} {"az.sdk.message":"Setting next AMQP channel.","connectionId":"MF_dae54d_1682077900713","entityPath":"$cbs"}
2023-04-21 13:51:43,075 INFO  [com.azu.cor.amq.imp.ReactorConnection] (reactor-executor-1) {} {"az.sdk.message":"Emitting new response channel.","connectionId":"MF_dae54d_1682077900713","entityPath":"$cbs","linkName":"cbs"}
2023-04-21 13:51:43,067 INFO  [com.azu.cor.amq.imp.han.SessionHandler] (reactor-executor-1) {} {"az.sdk.message":"onSessionRemoteOpen","connectionId":"MF_dae54d_1682077900713","sessionName":"cbs-session","sessionIncCapacity":0,"sessionOutgoingWindow":2147483647}
2023-04-21 13:51:43,034 INFO  [com.azu.cor.amq.imp.ReactorConnection] (reactor-executor-1) {} {"az.sdk.message":"Setting CBS channel.","connectionId":"MF_dae54d_1682077900713"}
2023-04-21 13:51:43,023 INFO  [com.azu.cor.amq.imp.han.SessionHandler] (reactor-executor-1) {} {"az.sdk.message":"onSessionRemoteOpen","connectionId":"MF_dae54d_1682077900713","sessionName":"generate_history","sessionIncCapacity":0,"sessionOutgoingWindow":2147483647}
2023-04-21 13:51:42,974 INFO  [com.azu.mes.ser.imp.ServiceBusConnectionProcessor] (reactor-executor-1) {} {"az.sdk.message":"Channel is now active.","entityPath":"N/A"}
2023-04-21 13:51:42,974 INFO  [com.azu.cor.amq.imp.han.WebSocketsConnectionHandler] (reactor-executor-1) {} {"az.sdk.message":"onConnectionRemoteOpen","connectionId":"MF_dae54d_1682077900713","hostName":"myservicebus.servicebus.windows.net","remoteContainer":"b885d5a5383f454db026bc00e6a84e14_G4S2"}
2023-04-21 13:51:42,830 INFO  [io.quarkus] (main) {} Installed features: [agroal, cache, cdi, cxf, hibernate-orm, jdbc-db2, micrometer, narayana-jta, opentelemetry, opentelemetry-otlp-exporter, reactive-routes, rest-client-reactive, rest-client-reactive-jackson, resteasy-reactive, resteasy-reactive-jackson, security, smallrye-context-propagation, smallrye-fault-tolerance, smallrye-health, smallrye-jwt, smallrye-openapi, smallrye-reactive-messaging, swagger-ui, vertx]
2023-04-21 13:51:42,830 INFO  [io.quarkus] (main) {} Profile prod activated. 
2023-04-21 13:51:42,830 INFO  [io.quarkus] (main) {} wealth-items-service a96a4b2c49 on JVM (powered by Quarkus 2.16.4.Final) started in 18.805s. Listening on: http://0.0.0.0:8080
2023-04-21 13:51:41,421 INFO  [com.azu.cor.amq.imp.han.WebSocketsConnectionHandler] (reactor-executor-1) {} {"az.sdk.message":"Adding web socket layer","connectionId":"MF_dae54d_1682077900713"}
2023-04-21 13:51:41,420 INFO  [com.azu.cor.amq.imp.han.WebSocketsConnectionHandler] (reactor-executor-1) {} {"az.sdk.message":"onConnectionBound","connectionId":"MF_dae54d_1682077900713","hostName":"myservicebus.servicebus.windows.net","peerDetails":"myservicebus.servicebus.windows.net:443"}
2023-04-21 13:51:41,325 INFO  [com.azu.mes.ser.ServiceBusClientBuilder] (main) {} # of open clients with shared connection: 1
2023-04-21 13:51:41,324 INFO  [com.azu.mes.ser.imp.ServiceBusConnectionProcessor] (main) {} {"az.sdk.message":"Setting next AMQP channel.","entityPath":"N/A"}
2023-04-21 13:51:41,323 INFO  [com.azu.cor.amq.ProxyOptions] (main) {} Username or password is null. Using system-wide authentication.
2023-04-21 13:51:41,116 INFO  [com.azu.cor.amq.imp.han.WebSocketsConnectionHandler] (reactor-executor-1) {} {"az.sdk.message":"onConnectionLocalOpen","connectionId":"MF_dae54d_1682077900713","errorCondition":null,"errorDescription":null,"hostName":"myservicebus.servicebus.windows.net"}
2023-04-21 13:51:41,115 INFO  [com.azu.cor.amq.imp.han.ReactorHandler] (reactor-executor-1) {} {"az.sdk.message":"reactor.onReactorInit","connectionId":"MF_dae54d_1682077900713"}
2023-04-21 13:51:41,114 INFO  [com.azu.cor.amq.imp.han.WebSocketsConnectionHandler] (reactor-executor-1) {} {"az.sdk.message":"onConnectionInit","connectionId":"MF_dae54d_1682077900713","hostName":"myservicebus.servicebus.windows.net","namespace":"myservicebus.servicebus.windows.net"}
2023-04-21 13:51:41,037 INFO  [com.azu.cor.amq.imp.ReactorExecutor] (main) {} {"az.sdk.message":"Starting reactor.","connectionId":"MF_dae54d_1682077900713"}
2023-04-21 13:51:41,022 INFO  [com.azu.cor.amq.imp.ReactorConnection] (main) {} {"az.sdk.message":"Creating and starting connection.","connectionId":"MF_dae54d_1682077900713","hostName":"myservicebus.servicebus.windows.net","port":443}
2023-04-21 13:51:40,930 INFO  [com.azu.mes.ser.imp.ServiceBusReceiveLinkProcessor] (main) {} Requesting a new AmqpReceiveLink from upstream.
2023-04-21 13:51:40,924 INFO  [com.azu.mes.ser.ServiceBusReceiverAsyncClient] (main) {} {"az.sdk.message":"Creating consumer.","linkName":"generate_history_359c54_1682077900924","entityPath":"generate_history"}
2023-04-21 13:51:40,835 INFO  [com.azu.mes.ser.ServiceBusClientBuilder] (main) {} # of open clients with shared connection: 1
2023-04-21 13:51:40,832 INFO  [com.azu.mes.ser.imp.ServiceBusConnectionProcessor] (main) {} {"az.sdk.message":"Setting next AMQP channel.","entityPath":"N/A"}
2023-04-21 13:51:40,424 INFO  [com.azu.cor.amq.ProxyOptions] (main) {} Username or password is null. Using system-wide authentication.
2023-04-21 13:51:39,229 INFO  [io.qua.sma.ope.run.OpenApiRecorder] (main) {} CORS filtering is disabled and cross-origin resource sharing is allowed without restriction, which is not recommended in production. Please configure the CORS filter through 'quarkus.http.cors.*' properties. For more information, see Quarkus HTTP CORS documentation
2023-04-21 13:51:38,824 INFO  [io.mic.cor.ins.pus.PushMeterRegistry] (main) {} publishing metrics for OtlpMeterRegistry every 1m
2023-04-21 13:51:27,114 WARN  [io.qua.config] (main) {} Unrecognized configuration key "quarkus.cxf.client.timeout" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/   
 -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \   
 --/ __ \/ / / / _ | / _ \/ //_/ / / / __/ 
__  ____  __  _____   ___  __ ____  ______ 
Picked up JAVA_TOOL_OPTIONS: -XX:+UseG1GC -XX:+UseContainerSupport -XX:MinRAMPercentage=50.0 -XX:MaxRAMPercentage=25.0

@AndreasPetersen
Copy link
Author

Update on the issue.

We have two different services. One receiving messages, one sending messages. When then service that stopped receiving messages, as per the error described in this issue, we had made a change to the sending service, where we started sending batch messages. However, we had made an error where we sent a list of messages as a single message, instead of a batch message. For some unknown reason, the receiving service stopped receiving messages at the same time in our production environment. In our testing environment, the receiving service continued to receive messages, but they would just fail at being mapped, due to the wrong data type, but that's just our fault.

We have tried restarting both the sending and receiving service many times, without any change to the situation. We then tried deleting the queue and creating a new one. Both the sending and receiving service threw errors during the downtime that they could not find the queue, as would be expected. When the queue was recreated, the sending service automatically reconnected and resumed sending messages to the queue. The receiving service stopped throwing the error described in this issue, but did not receive messages. After restarting the receiving service, it now receives messages as normal again.

We have no idea why the change in sending messages would result in messages not being received in one environment (production), but not another (test). Likewise, we have no idea why it started working again.

@AndreasPetersen AndreasPetersen mentioned this issue Jul 31, 2023
Closed
@AndreasPetersen
Copy link
Author

AndreasPetersen commented Jul 31, 2023 •
edited
Loading

Update on the issue.

We discovered a bug in our usage of the Flux returned from .receiveMessages() on the async client, resulting in up to 256 messages being received in LOCK_RECEIVE_MODE, however we would not process them in time. This resulted in a bunch of lock renewels. However, I fail to understand why this would cause the force detached issue.

Regardless, we solved the above bug, and have since gotten a much more stable usage. However, we still experience the error about 1 time per week. The error does not resolve itself, unless we restart the service. Coupled with the fact that there is no health indicator on the client (#21976), we have to manually start the service, which is pretty bad.

Since the initial bug report, we have also updated our Azure dependencies, and are currently using the following versions:

gradle dependencies -q | grep azure
|    +--- com.azure:azure-sdk-bom:1.2.14
+--- com.azure:azure-sdk-bom -> 1.2.14
+--- dk.bankdata.qua:azure-webpubsub-tools:1.0.0
+--- com.azure:azure-sdk-bom (n)
+--- dk.bankdata.qua:azure-webpubsub-tools:1.0.0 (n)
|    +--- com.azure:azure-sdk-bom:1.2.14
+--- com.azure:azure-sdk-bom -> 1.2.14
+--- dk.bankdata.qua:azure-webpubsub-tools:1.0.0
|    +--- com.azure:azure-sdk-bom:1.2.14
|    |    +--- com.azure:azure-messaging-webpubsub:1.2.5 (c)
|    |    +--- com.azure:azure-core:1.40.0 (c)
|    |    +--- com.azure:azure-core-http-netty:1.13.4 (c)
|    |    +--- com.azure:azure-messaging-servicebus:7.14.1 (c)
|    |    +--- com.azure:azure-identity:1.9.1 (c)
|    |    +--- com.azure:azure-json:1.0.1 (c)
|    |    \--- com.azure:azure-core-amqp:2.8.6 (c)
|    +--- com.azure:azure-core-tracing-opentelemetry:1.0.0-beta.36 (c)
|    \--- com.microsoft.azure:msal4j:1.13.8 (c)
+--- com.azure:azure-sdk-bom -> 1.2.14 (*)
|    |    +--- com.azure:azure-core:1.37.0 -> 1.40.0 (c)
|    |    +--- com.azure:azure-messaging-servicebus:7.13.3 -> 7.14.1 (c)
|    |    +--- com.azure:azure-identity:1.8.1 -> 1.9.1 (c)
|    |    +--- com.azure:azure-core-amqp:2.8.3 -> 2.8.6 (c)
|    |    +--- com.microsoft.azure:msal4j:1.13.5 -> 1.13.8 (c)
|         +--- com.azure:azure-core-tracing-opentelemetry -> 1.0.0-beta.36
|         |    \--- com.azure:azure-core:1.40.0
|         |         +--- com.azure:azure-json:1.0.1
|         +--- com.azure:azure-messaging-servicebus -> 7.14.1
|         |    +--- com.azure:azure-core:1.40.0 (*)
|         |    +--- com.azure:azure-core-amqp:2.8.6
|         |    |    +--- com.azure:azure-core:1.40.0 (*)
|         |    |    +--- com.microsoft.azure:qpid-proton-j-extensions:1.2.4
|         |    +--- com.azure:azure-core-http-netty:1.13.4
|         |    |    +--- com.azure:azure-core:1.40.0 (*)
|         +--- com.azure:azure-identity -> 1.9.1
|         |    +--- com.azure:azure-core:1.40.0 (*)
|         |    +--- com.azure:azure-core-http-netty:1.13.4 (*)
|         |    +--- com.microsoft.azure:msal4j:1.13.8
|         |    +--- com.microsoft.azure:msal4j-persistence-extension:1.2.0
|         |    |    +--- com.microsoft.azure:msal4j:1.4.0 -> 1.13.8 (*)
+--- dk.bankdata.qua:azure-webpubsub-tools:1.0.0
|    +--- com.azure:azure-core-tracing-opentelemetry:1.0.0-beta.36 (*)
|    +--- com.azure:azure-messaging-webpubsub:1.2.4 -> 1.2.5
|    |    +--- com.azure:azure-core:1.40.0 (*)
|    |    +--- com.azure:azure-core-http-netty:1.13.4 (*)
|    +--- com.azure:azure-sdk-bom:1.2.14
|    |    +--- com.azure:azure-messaging-webpubsub:1.2.5 (c)
|    |    +--- com.azure:azure-core:1.40.0 (c)
|    |    +--- com.azure:azure-core-http-netty:1.13.4 (c)
|    |    +--- com.azure:azure-messaging-servicebus:7.14.1 (c)
|    |    +--- com.azure:azure-identity:1.9.1 (c)
|    |    +--- com.azure:azure-json:1.0.1 (c)
|    |    \--- com.azure:azure-core-amqp:2.8.6 (c)
|    +--- com.azure:azure-core-tracing-opentelemetry:1.0.0-beta.36 (c)
|    \--- com.microsoft.azure:msal4j:1.13.8 (c)
+--- com.azure:azure-sdk-bom -> 1.2.14 (*)
|    |    +--- com.azure:azure-core:1.37.0 -> 1.40.0 (c)
|    |    +--- com.azure:azure-messaging-servicebus:7.13.3 -> 7.14.1 (c)
|    |    +--- com.azure:azure-identity:1.8.1 -> 1.9.1 (c)
|    |    +--- com.azure:azure-core-amqp:2.8.3 -> 2.8.6 (c)
|    |    +--- com.microsoft.azure:msal4j:1.13.5 -> 1.13.8 (c)
|         +--- com.azure:azure-core-tracing-opentelemetry -> 1.0.0-beta.36
|         |    \--- com.azure:azure-core:1.40.0
|         |         +--- com.azure:azure-json:1.0.1
|         +--- com.azure:azure-messaging-servicebus -> 7.14.1
|         |    +--- com.azure:azure-core:1.40.0 (*)
|         |    +--- com.azure:azure-core-amqp:2.8.6
|         |    |    +--- com.azure:azure-core:1.40.0 (*)
|         |    |    +--- com.microsoft.azure:qpid-proton-j-extensions:1.2.4
|         |    +--- com.azure:azure-core-http-netty:1.13.4
|         |    |    +--- com.azure:azure-core:1.40.0 (*)
|         +--- com.azure:azure-identity -> 1.9.1
|         |    +--- com.azure:azure-core:1.40.0 (*)
|         |    +--- com.azure:azure-core-http-netty:1.13.4 (*)
|         |    +--- com.microsoft.azure:msal4j:1.13.8
|         |    +--- com.microsoft.azure:msal4j-persistence-extension:1.2.0
|         |    |    +--- com.microsoft.azure:msal4j:1.4.0 -> 1.13.8 (*)
+--- dk.bankdata.qua:azure-webpubsub-tools:1.0.0
|    +--- com.azure:azure-core-tracing-opentelemetry:1.0.0-beta.36 (*)
|    +--- com.azure:azure-messaging-webpubsub:1.2.4 -> 1.2.5
|    |    +--- com.azure:azure-core:1.40.0 (*)
|    |    +--- com.azure:azure-core-http-netty:1.13.4 (*)
|    +--- com.azure:azure-sdk-bom:1.2.14
+--- com.azure:azure-sdk-bom -> 1.2.14
+--- dk.bankdata.qua:azure-webpubsub-tools:1.0.0
|    +--- com.azure:azure-sdk-bom:1.2.14
|    |    +--- com.azure:azure-messaging-webpubsub:1.2.5 (c)
|    |    +--- com.azure:azure-core:1.40.0 (c)
|    |    +--- com.azure:azure-core-http-netty:1.13.4 (c)
|    |    +--- com.azure:azure-messaging-servicebus:7.14.1 (c)
|    |    +--- com.azure:azure-identity:1.9.1 (c)
|    |    +--- com.azure:azure-json:1.0.1 (c)
|    |    \--- com.azure:azure-core-amqp:2.8.6 (c)
|    +--- com.azure:azure-core-tracing-opentelemetry:1.0.0-beta.36 (c)
|    \--- com.microsoft.azure:msal4j:1.13.8 (c)
+--- com.azure:azure-sdk-bom -> 1.2.14 (*)
|    |    +--- com.azure:azure-core:1.37.0 -> 1.40.0 (c)
|    |    +--- com.azure:azure-messaging-servicebus:7.13.3 -> 7.14.1 (c)
|    |    +--- com.azure:azure-identity:1.8.1 -> 1.9.1 (c)
|    |    +--- com.azure:azure-core-amqp:2.8.3 -> 2.8.6 (c)
|    |    +--- com.microsoft.azure:msal4j:1.13.5 -> 1.13.8 (c)
|         +--- com.azure:azure-core-tracing-opentelemetry -> 1.0.0-beta.36
|         |    \--- com.azure:azure-core:1.40.0
|         |         +--- com.azure:azure-json:1.0.1
|         +--- com.azure:azure-messaging-servicebus -> 7.14.1
|         |    +--- com.azure:azure-core:1.40.0 (*)
|         |    +--- com.azure:azure-core-amqp:2.8.6
|         |    |    +--- com.azure:azure-core:1.40.0 (*)
|         |    |    +--- com.microsoft.azure:qpid-proton-j-extensions:1.2.4
|         |    +--- com.azure:azure-core-http-netty:1.13.4
|         |    |    +--- com.azure:azure-core:1.40.0 (*)
|         +--- com.azure:azure-identity -> 1.9.1
|         |    +--- com.azure:azure-core:1.40.0 (*)
|         |    +--- com.azure:azure-core-http-netty:1.13.4 (*)
|         |    +--- com.microsoft.azure:msal4j:1.13.8
|         |    +--- com.microsoft.azure:msal4j-persistence-extension:1.2.0
|         |    |    +--- com.microsoft.azure:msal4j:1.4.0 -> 1.13.8 (*)
+--- dk.bankdata.qua:azure-webpubsub-tools:1.0.0
|    +--- com.azure:azure-core-tracing-opentelemetry:1.0.0-beta.36 (*)
|    +--- com.azure:azure-messaging-webpubsub:1.2.4 -> 1.2.5
|    |    +--- com.azure:azure-core:1.40.0 (*)
|    |    +--- com.azure:azure-core-http-netty:1.13.4 (*)
|    +--- com.azure:azure-sdk-bom:1.2.14
|    |    +--- com.azure:azure-messaging-webpubsub:1.2.5 (c)
|    |    +--- com.azure:azure-core:1.40.0 (c)
|    |    +--- com.azure:azure-core-http-netty:1.13.4 (c)
|    |    +--- com.azure:azure-messaging-servicebus:7.14.1 (c)
|    |    +--- com.azure:azure-identity:1.9.1 (c)
|    |    +--- com.azure:azure-json:1.0.1 (c)
|    |    \--- com.azure:azure-core-amqp:2.8.6 (c)
|    +--- com.azure:azure-core-tracing-opentelemetry:1.0.0-beta.36 (c)
|    \--- com.microsoft.azure:msal4j:1.13.8 (c)
+--- com.azure:azure-sdk-bom -> 1.2.14 (*)
|    |    +--- com.azure:azure-core:1.37.0 -> 1.40.0 (c)
|    |    +--- com.azure:azure-messaging-servicebus:7.13.3 -> 7.14.1 (c)
|    |    +--- com.azure:azure-identity:1.8.1 -> 1.9.1 (c)
|    |    +--- com.azure:azure-core-amqp:2.8.3 -> 2.8.6 (c)
|    |    +--- com.microsoft.azure:msal4j:1.13.5 -> 1.13.8 (c)
|         +--- com.azure:azure-core-tracing-opentelemetry -> 1.0.0-beta.36
|         |    \--- com.azure:azure-core:1.40.0
|         |         +--- com.azure:azure-json:1.0.1
|         +--- com.azure:azure-messaging-servicebus -> 7.14.1
|         |    +--- com.azure:azure-core:1.40.0 (*)
|         |    +--- com.azure:azure-core-amqp:2.8.6
|         |    |    +--- com.azure:azure-core:1.40.0 (*)
|         |    |    +--- com.microsoft.azure:qpid-proton-j-extensions:1.2.4
|         |    +--- com.azure:azure-core-http-netty:1.13.4
|         |    |    +--- com.azure:azure-core:1.40.0 (*)
|         +--- com.azure:azure-identity -> 1.9.1
|         |    +--- com.azure:azure-core:1.40.0 (*)
|         |    +--- com.azure:azure-core-http-netty:1.13.4 (*)
|         |    +--- com.microsoft.azure:msal4j:1.13.8
|         |    +--- com.microsoft.azure:msal4j-persistence-extension:1.2.0
|         |    |    +--- com.microsoft.azure:msal4j:1.4.0 -> 1.13.8 (*)
+--- dk.bankdata.qua:azure-webpubsub-tools:1.0.0
|    +--- com.azure:azure-core-tracing-opentelemetry:1.0.0-beta.36 (*)
|    +--- com.azure:azure-messaging-webpubsub:1.2.4 -> 1.2.5
|    |    +--- com.azure:azure-core:1.40.0 (*)
|    |    +--- com.azure:azure-core-http-netty:1.13.4 (*)
|    +--- com.azure:azure-sdk-bom:1.2.14
|    |    +--- com.azure:azure-messaging-webpubsub:1.2.5 (c)
|    |    +--- com.azure:azure-core:1.40.0 (c)
|    |    +--- com.azure:azure-core-http-netty:1.13.4 (c)
|    |    +--- com.azure:azure-messaging-servicebus:7.14.1 (c)
|    |    +--- com.azure:azure-identity:1.9.1 (c)
|    |    +--- com.azure:azure-json:1.0.1 (c)
|    |    \--- com.azure:azure-core-amqp:2.8.6 (c)
|    +--- com.azure:azure-core-tracing-opentelemetry:1.0.0-beta.36 (c)
|    \--- com.microsoft.azure:msal4j:1.13.8 (c)
+--- com.azure:azure-sdk-bom -> 1.2.14 (*)
|    |    +--- com.azure:azure-core:1.37.0 -> 1.40.0 (c)
|    |    +--- com.azure:azure-messaging-servicebus:7.13.3 -> 7.14.1 (c)
|    |    +--- com.azure:azure-identity:1.8.1 -> 1.9.1 (c)
|    |    +--- com.azure:azure-core-amqp:2.8.3 -> 2.8.6 (c)
|    |    +--- com.microsoft.azure:msal4j:1.13.5 -> 1.13.8 (c)
|         +--- com.azure:azure-core-tracing-opentelemetry -> 1.0.0-beta.36
|         |    \--- com.azure:azure-core:1.40.0
|         |         +--- com.azure:azure-json:1.0.1
|         +--- com.azure:azure-messaging-servicebus -> 7.14.1
|         |    +--- com.azure:azure-core:1.40.0 (*)
|         |    +--- com.azure:azure-core-amqp:2.8.6
|         |    |    +--- com.azure:azure-core:1.40.0 (*)
|         |    |    +--- com.microsoft.azure:qpid-proton-j-extensions:1.2.4
|         |    +--- com.azure:azure-core-http-netty:1.13.4
|         |    |    +--- com.azure:azure-core:1.40.0 (*)
|         +--- com.azure:azure-identity -> 1.9.1
|         |    +--- com.azure:azure-core:1.40.0 (*)
|         |    +--- com.azure:azure-core-http-netty:1.13.4 (*)
|         |    +--- com.microsoft.azure:msal4j:1.13.8
|         |    +--- com.microsoft.azure:msal4j-persistence-extension:1.2.0
|         |    |    +--- com.microsoft.azure:msal4j:1.4.0 -> 1.13.8 (*)
+--- dk.bankdata.qua:azure-webpubsub-tools:1.0.0
|    +--- com.azure:azure-core-tracing-opentelemetry:1.0.0-beta.36 (*)
|    +--- com.azure:azure-messaging-webpubsub:1.2.4 -> 1.2.5
|    |    +--- com.azure:azure-core:1.40.0 (*)
|    |    +--- com.azure:azure-core-http-netty:1.13.4 (*)
|    +--- com.azure:azure-sdk-bom:1.2.14
|    |    +--- com.azure:azure-messaging-webpubsub:1.2.5 (c)
|    |    +--- com.azure:azure-core:1.40.0 (c)
|    |    +--- com.azure:azure-core-http-netty:1.13.4 (c)
|    |    +--- com.azure:azure-messaging-servicebus:7.14.1 (c)
|    |    +--- com.azure:azure-identity:1.9.1 (c)
|    |    +--- com.azure:azure-json:1.0.1 (c)
|    |    \--- com.azure:azure-core-amqp:2.8.6 (c)
|    +--- com.azure:azure-core-tracing-opentelemetry:1.0.0-beta.36 (c)
|    \--- com.microsoft.azure:msal4j:1.13.8 (c)
+--- com.azure:azure-sdk-bom -> 1.2.14 (*)
|    |    +--- com.azure:azure-core:1.37.0 -> 1.40.0 (c)
|    |    +--- com.azure:azure-messaging-servicebus:7.13.3 -> 7.14.1 (c)
|    |    +--- com.azure:azure-identity:1.8.1 -> 1.9.1 (c)
|    |    +--- com.azure:azure-core-amqp:2.8.3 -> 2.8.6 (c)
|    |    +--- com.microsoft.azure:msal4j:1.13.5 -> 1.13.8 (c)
|         +--- com.azure:azure-core-tracing-opentelemetry -> 1.0.0-beta.36
|         |    \--- com.azure:azure-core:1.40.0
|         |         +--- com.azure:azure-json:1.0.1
|         +--- com.azure:azure-messaging-servicebus -> 7.14.1
|         |    +--- com.azure:azure-core:1.40.0 (*)
|         |    +--- com.azure:azure-core-amqp:2.8.6
|         |    |    +--- com.azure:azure-core:1.40.0 (*)
|         |    |    +--- com.microsoft.azure:qpid-proton-j-extensions:1.2.4
|         |    +--- com.azure:azure-core-http-netty:1.13.4
|         |    |    +--- com.azure:azure-core:1.40.0 (*)
|         +--- com.azure:azure-identity -> 1.9.1
|         |    +--- com.azure:azure-core:1.40.0 (*)
|         |    +--- com.azure:azure-core-http-netty:1.13.4 (*)
|         |    +--- com.microsoft.azure:msal4j:1.13.8
|         |    +--- com.microsoft.azure:msal4j-persistence-extension:1.2.0
|         |    |    +--- com.microsoft.azure:msal4j:1.4.0 -> 1.13.8 (*)
+--- dk.bankdata.qua:azure-webpubsub-tools:1.0.0
|    +--- com.azure:azure-core-tracing-opentelemetry:1.0.0-beta.36 (*)
|    +--- com.azure:azure-messaging-webpubsub:1.2.4 -> 1.2.5
|    |    +--- com.azure:azure-core:1.40.0 (*)
|    |    +--- com.azure:azure-core-http-netty:1.13.4 (*)
|    +--- com.azure:azure-sdk-bom:1.2.14
|    |    +--- com.azure:azure-messaging-webpubsub:1.2.5 (c)
|    |    +--- com.azure:azure-core:1.40.0 (c)
|    |    +--- com.azure:azure-core-http-netty:1.13.4 (c)
|    |    +--- com.azure:azure-messaging-servicebus:7.14.1 (c)
|    |    +--- com.azure:azure-identity:1.9.1 (c)
|    |    +--- com.azure:azure-json:1.0.1 (c)
|    |    \--- com.azure:azure-core-amqp:2.8.6 (c)
|    +--- com.azure:azure-core-tracing-opentelemetry:1.0.0-beta.36 (c)
|    \--- com.microsoft.azure:msal4j:1.13.8 (c)
+--- com.azure:azure-sdk-bom -> 1.2.14 (*)
|    |    +--- com.azure:azure-core:1.37.0 -> 1.40.0 (c)
|    |    +--- com.azure:azure-messaging-servicebus:7.13.3 -> 7.14.1 (c)
|    |    +--- com.azure:azure-identity:1.8.1 -> 1.9.1 (c)
|    |    +--- com.azure:azure-core-amqp:2.8.3 -> 2.8.6 (c)
|    |    +--- com.microsoft.azure:msal4j:1.13.5 -> 1.13.8 (c)
|         +--- com.azure:azure-core-tracing-opentelemetry -> 1.0.0-beta.36
|         |    \--- com.azure:azure-core:1.40.0
|         |         +--- com.azure:azure-json:1.0.1
|         +--- com.azure:azure-messaging-servicebus -> 7.14.1
|         |    +--- com.azure:azure-core:1.40.0 (*)
|         |    +--- com.azure:azure-core-amqp:2.8.6
|         |    |    +--- com.azure:azure-core:1.40.0 (*)
|         |    |    +--- com.microsoft.azure:qpid-proton-j-extensions:1.2.4
|         |    +--- com.azure:azure-core-http-netty:1.13.4
|         |    |    +--- com.azure:azure-core:1.40.0 (*)
|         +--- com.azure:azure-identity -> 1.9.1
|         |    +--- com.azure:azure-core:1.40.0 (*)
|         |    +--- com.azure:azure-core-http-netty:1.13.4 (*)
|         |    +--- com.microsoft.azure:msal4j:1.13.8
|         |    +--- com.microsoft.azure:msal4j-persistence-extension:1.2.0
|         |    |    +--- com.microsoft.azure:msal4j:1.4.0 -> 1.13.8 (*)
+--- dk.bankdata.qua:azure-webpubsub-tools:1.0.0
|    +--- com.azure:azure-core-tracing-opentelemetry:1.0.0-beta.36 (*)
|    +--- com.azure:azure-messaging-webpubsub:1.2.4 -> 1.2.5
|    |    +--- com.azure:azure-core:1.40.0 (*)
|    |    +--- com.azure:azure-core-http-netty:1.13.4 (*)
|    |    |    +--- com.azure:azure-messaging-servicebus:7.13.3 -> 7.14.1 (c)
|    |    |    +--- com.azure:azure-identity:1.8.1 -> 1.9.1 (c)
|    |    |    +--- com.azure:azure-core:1.37.0 -> 1.40.0 (c)
|    |    |    +--- com.azure:azure-core-amqp:2.8.3 -> 2.8.6 (c)
|    |    |    +--- com.microsoft.azure:msal4j:1.13.5 -> 1.13.8 (c)
|    |         |    +--- com.azure:azure-sdk-bom:1.2.14
|    |         |    |    +--- com.azure:azure-messaging-servicebus:7.14.1 (c)
|    |         |    |    +--- com.azure:azure-identity:1.9.1 (c)
|    |         |    |    +--- com.azure:azure-core:1.40.0 (c)
|    |         |    |    +--- com.azure:azure-core-amqp:2.8.6 (c)
|    |         |    |    +--- com.azure:azure-core-http-netty:1.13.4 (c)
|    |         |    |    \--- com.azure:azure-json:1.0.1 (c)
|    |         |    +--- com.azure:azure-core-tracing-opentelemetry:1.0.0-beta.36 (c)
|    |         |    \--- com.microsoft.azure:msal4j:1.13.8 (c)
|    |         +--- com.azure:azure-core-tracing-opentelemetry -> 1.0.0-beta.36
|    |         |    \--- com.azure:azure-core:1.40.0
|    |         |         +--- com.azure:azure-json:1.0.1
|    |         +--- com.azure:azure-messaging-servicebus -> 7.14.1
|    |         |    +--- com.azure:azure-core:1.40.0 (*)
|    |         |    +--- com.azure:azure-core-amqp:2.8.6
|    |         |    |    +--- com.azure:azure-core:1.40.0 (*)
|    |         |    |    +--- com.microsoft.azure:qpid-proton-j-extensions:1.2.4
|    |         |    +--- com.azure:azure-core-http-netty:1.13.4
|    |         |    |    +--- com.azure:azure-core:1.40.0 (*)
|    |         +--- com.azure:azure-identity -> 1.9.1
|    |         |    +--- com.azure:azure-core:1.40.0 (*)
|    |         |    +--- com.azure:azure-core-http-netty:1.13.4 (*)
|    |         |    +--- com.microsoft.azure:msal4j:1.13.8
|    |         |    +--- com.microsoft.azure:msal4j-persistence-extension:1.2.0
|    |         |    |    +--- com.microsoft.azure:msal4j:1.4.0 -> 1.13.8 (*)
\--- com.azure:azure-sdk-bom -> 1.2.14 (*)
|    \--- com.azure:azure-sdk-bom:1.2.14
\--- com.azure:azure-sdk-bom -> 1.2.14
|    +--- com.azure:azure-sdk-bom:1.2.14
|    |    +--- com.azure:azure-messaging-webpubsub:1.2.5 (c)
|    |    +--- com.azure:azure-core:1.40.0 (c)
|    |    +--- com.azure:azure-core-http-netty:1.13.4 (c)
|    |    +--- com.azure:azure-messaging-servicebus:7.14.1 (c)
|    |    +--- com.azure:azure-identity:1.9.1 (c)
|    |    +--- com.azure:azure-json:1.0.1 (c)
|    |    \--- com.azure:azure-core-amqp:2.8.6 (c)
|    +--- com.azure:azure-core-tracing-opentelemetry:1.0.0-beta.36 (c)
|    \--- com.microsoft.azure:msal4j:1.13.8 (c)
+--- com.azure:azure-sdk-bom -> 1.2.14 (*)
|    |    +--- com.azure:azure-core:1.37.0 -> 1.40.0 (c)
|    |    +--- com.azure:azure-messaging-servicebus:7.13.3 -> 7.14.1 (c)
|    |    +--- com.azure:azure-identity:1.8.1 -> 1.9.1 (c)
|    |    +--- com.azure:azure-core-amqp:2.8.3 -> 2.8.6 (c)
|    |    +--- com.microsoft.azure:msal4j:1.13.5 -> 1.13.8 (c)
|         +--- com.azure:azure-core-tracing-opentelemetry -> 1.0.0-beta.36
|         |    \--- com.azure:azure-core:1.40.0
|         |         +--- com.azure:azure-json:1.0.1
|         +--- com.azure:azure-messaging-servicebus -> 7.14.1
|         |    +--- com.azure:azure-core:1.40.0 (*)
|         |    +--- com.azure:azure-core-amqp:2.8.6
|         |    |    +--- com.azure:azure-core:1.40.0 (*)
|         |    |    +--- com.microsoft.azure:qpid-proton-j-extensions:1.2.4
|         |    +--- com.azure:azure-core-http-netty:1.13.4
|         |    |    +--- com.azure:azure-core:1.40.0 (*)
|         +--- com.azure:azure-identity -> 1.9.1
|         |    +--- com.azure:azure-core:1.40.0 (*)
|         |    +--- com.azure:azure-core-http-netty:1.13.4 (*)
|         |    +--- com.microsoft.azure:msal4j:1.13.8
|         |    +--- com.microsoft.azure:msal4j-persistence-extension:1.2.0
|         |    |    +--- com.microsoft.azure:msal4j:1.4.0 -> 1.13.8 (*)
+--- dk.bankdata.qua:azure-webpubsub-tools:1.0.0
|    +--- com.azure:azure-core-tracing-opentelemetry:1.0.0-beta.36 (*)
|    +--- com.azure:azure-messaging-webpubsub:1.2.4 -> 1.2.5
|    |    +--- com.azure:azure-core:1.40.0 (*)
|    |    +--- com.azure:azure-core-http-netty:1.13.4 (*)
|    +--- com.azure:azure-sdk-bom:1.2.14
|    |    +--- com.azure:azure-messaging-webpubsub:1.2.5 (c)
|    |    +--- com.azure:azure-core:1.40.0 (c)
|    |    +--- com.azure:azure-core-http-netty:1.13.4 (c)
|    |    +--- com.azure:azure-messaging-servicebus:7.14.1 (c)
|    |    +--- com.azure:azure-identity:1.9.1 (c)
|    |    +--- com.azure:azure-json:1.0.1 (c)
|    |    \--- com.azure:azure-core-amqp:2.8.6 (c)
|    +--- com.azure:azure-core-tracing-opentelemetry:1.0.0-beta.36 (c)
|    \--- com.microsoft.azure:msal4j:1.13.8 (c)
+--- com.azure:azure-sdk-bom -> 1.2.14 (*)
|    |    +--- com.azure:azure-core:1.37.0 -> 1.40.0 (c)
|    |    +--- com.azure:azure-messaging-servicebus:7.13.3 -> 7.14.1 (c)
|    |    +--- com.azure:azure-identity:1.8.1 -> 1.9.1 (c)
|    |    +--- com.azure:azure-core-amqp:2.8.3 -> 2.8.6 (c)
|    |    +--- com.microsoft.azure:msal4j:1.13.5 -> 1.13.8 (c)
|         +--- com.azure:azure-core-tracing-opentelemetry -> 1.0.0-beta.36
|         |    \--- com.azure:azure-core:1.40.0
|         |         +--- com.azure:azure-json:1.0.1
|         +--- com.azure:azure-messaging-servicebus -> 7.14.1
|         |    +--- com.azure:azure-core:1.40.0 (*)
|         |    +--- com.azure:azure-core-amqp:2.8.6
|         |    |    +--- com.azure:azure-core:1.40.0 (*)
|         |    |    +--- com.microsoft.azure:qpid-proton-j-extensions:1.2.4
|         |    +--- com.azure:azure-core-http-netty:1.13.4
|         |    |    +--- com.azure:azure-core:1.40.0 (*)
|         +--- com.azure:azure-identity -> 1.9.1
|         |    +--- com.azure:azure-core:1.40.0 (*)
|         |    +--- com.azure:azure-core-http-netty:1.13.4 (*)
|         |    +--- com.microsoft.azure:msal4j:1.13.8
|         |    +--- com.microsoft.azure:msal4j-persistence-extension:1.2.0
|         |    |    +--- com.microsoft.azure:msal4j:1.4.0 -> 1.13.8 (*)
+--- dk.bankdata.qua:azure-webpubsub-tools:1.0.0
|    +--- com.azure:azure-core-tracing-opentelemetry:1.0.0-beta.36 (*)
|    +--- com.azure:azure-messaging-webpubsub:1.2.4 -> 1.2.5
|    |    +--- com.azure:azure-core:1.40.0 (*)
|    |    +--- com.azure:azure-core-http-netty:1.13.4 (*)
|    |    |    +--- com.azure:azure-messaging-servicebus:7.13.3 -> 7.14.1 (c)
|    |    |    +--- com.azure:azure-identity:1.8.1 -> 1.9.1 (c)
|    |    |    +--- com.azure:azure-core:1.37.0 -> 1.40.0 (c)
|    |    |    +--- com.azure:azure-core-amqp:2.8.3 -> 2.8.6 (c)
|    |    |    +--- com.microsoft.azure:msal4j:1.13.5 -> 1.13.8 (c)
|    |         |    +--- com.azure:azure-sdk-bom:1.2.14
|    |         |    |    +--- com.azure:azure-messaging-servicebus:7.14.1 (c)
|    |         |    |    +--- com.azure:azure-identity:1.9.1 (c)
|    |         |    |    +--- com.azure:azure-core:1.40.0 (c)
|    |         |    |    +--- com.azure:azure-core-amqp:2.8.6 (c)
|    |         |    |    +--- com.azure:azure-core-http-netty:1.13.4 (c)
|    |         |    |    \--- com.azure:azure-json:1.0.1 (c)
|    |         |    +--- com.azure:azure-core-tracing-opentelemetry:1.0.0-beta.36 (c)
|    |         |    \--- com.microsoft.azure:msal4j:1.13.8 (c)
|    |         +--- com.azure:azure-core-tracing-opentelemetry -> 1.0.0-beta.36
|    |         |    \--- com.azure:azure-core:1.40.0
|    |         |         +--- com.azure:azure-json:1.0.1
|    |         +--- com.azure:azure-messaging-servicebus -> 7.14.1
|    |         |    +--- com.azure:azure-core:1.40.0 (*)
|    |         |    +--- com.azure:azure-core-amqp:2.8.6
|    |         |    |    +--- com.azure:azure-core:1.40.0 (*)
|    |         |    |    +--- com.microsoft.azure:qpid-proton-j-extensions:1.2.4
|    |         |    +--- com.azure:azure-core-http-netty:1.13.4
|    |         |    |    +--- com.azure:azure-core:1.40.0 (*)
|    |         +--- com.azure:azure-identity -> 1.9.1
|    |         |    +--- com.azure:azure-core:1.40.0 (*)
|    |         |    +--- com.azure:azure-core-http-netty:1.13.4 (*)
|    |         |    +--- com.microsoft.azure:msal4j:1.13.8
|    |         |    +--- com.microsoft.azure:msal4j-persistence-extension:1.2.0
|    |         |    |    +--- com.microsoft.azure:msal4j:1.4.0 -> 1.13.8 (*)
\--- com.azure:azure-sdk-bom -> 1.2.14 (*)
|    \--- com.azure:azure-sdk-bom:1.2.14
\--- com.azure:azure-sdk-bom -> 1.2.14
|    +--- com.azure:azure-sdk-bom:1.2.14
|    |    +--- com.azure:azure-messaging-webpubsub:1.2.5 (c)
|    |    +--- com.azure:azure-core:1.40.0 (c)
|    |    +--- com.azure:azure-core-http-netty:1.13.4 (c)
|    |    +--- com.azure:azure-messaging-servicebus:7.14.1 (c)
|    |    +--- com.azure:azure-identity:1.9.1 (c)
|    |    +--- com.azure:azure-json:1.0.1 (c)
|    |    \--- com.azure:azure-core-amqp:2.8.6 (c)
|    +--- com.azure:azure-core-tracing-opentelemetry:1.0.0-beta.36 (c)
|    \--- com.microsoft.azure:msal4j:1.13.8 (c)
+--- com.azure:azure-sdk-bom -> 1.2.14 (*)
|    |    +--- com.azure:azure-core:1.37.0 -> 1.40.0 (c)
|    |    +--- com.azure:azure-messaging-servicebus:7.13.3 -> 7.14.1 (c)
|    |    +--- com.azure:azure-identity:1.8.1 -> 1.9.1 (c)
|    |    +--- com.azure:azure-core-amqp:2.8.3 -> 2.8.6 (c)
|    |    +--- com.microsoft.azure:msal4j:1.13.5 -> 1.13.8 (c)
|         +--- com.azure:azure-core-tracing-opentelemetry -> 1.0.0-beta.36
|         |    \--- com.azure:azure-core:1.40.0
|         |         +--- com.azure:azure-json:1.0.1
|         +--- com.azure:azure-messaging-servicebus -> 7.14.1
|         |    +--- com.azure:azure-core:1.40.0 (*)
|         |    +--- com.azure:azure-core-amqp:2.8.6
|         |    |    +--- com.azure:azure-core:1.40.0 (*)
|         |    |    +--- com.microsoft.azure:qpid-proton-j-extensions:1.2.4
|         |    +--- com.azure:azure-core-http-netty:1.13.4
|         |    |    +--- com.azure:azure-core:1.40.0 (*)
|         +--- com.azure:azure-identity -> 1.9.1
|         |    +--- com.azure:azure-core:1.40.0 (*)
|         |    +--- com.azure:azure-core-http-netty:1.13.4 (*)
|         |    +--- com.microsoft.azure:msal4j:1.13.8
|         |    +--- com.microsoft.azure:msal4j-persistence-extension:1.2.0
|         |    |    +--- com.microsoft.azure:msal4j:1.4.0 -> 1.13.8 (*)
+--- dk.bankdata.qua:azure-webpubsub-tools:1.0.0
|    +--- com.azure:azure-core-tracing-opentelemetry:1.0.0-beta.36 (*)
|    +--- com.azure:azure-messaging-webpubsub:1.2.4 -> 1.2.5
|    |    +--- com.azure:azure-core:1.40.0 (*)
|    |    +--- com.azure:azure-core-http-netty:1.13.4 (*)
|    +--- com.azure:azure-sdk-bom:1.2.14
|    |    +--- com.azure:azure-messaging-webpubsub:1.2.5 (c)
|    |    +--- com.azure:azure-core:1.40.0 (c)
|    |    +--- com.azure:azure-core-http-netty:1.13.4 (c)
|    |    +--- com.azure:azure-messaging-servicebus:7.14.1 (c)
|    |    +--- com.azure:azure-identity:1.9.1 (c)
|    |    +--- com.azure:azure-json:1.0.1 (c)
|    |    \--- com.azure:azure-core-amqp:2.8.6 (c)
|    +--- com.azure:azure-core-tracing-opentelemetry:1.0.0-beta.36 (c)
|    \--- com.microsoft.azure:msal4j:1.13.8 (c)
+--- com.azure:azure-sdk-bom -> 1.2.14 (*)
|    |    +--- com.azure:azure-core:1.37.0 -> 1.40.0 (c)
|    |    +--- com.azure:azure-messaging-servicebus:7.13.3 -> 7.14.1 (c)
|    |    +--- com.azure:azure-identity:1.8.1 -> 1.9.1 (c)
|    |    +--- com.azure:azure-core-amqp:2.8.3 -> 2.8.6 (c)
|    |    +--- com.microsoft.azure:msal4j:1.13.5 -> 1.13.8 (c)
|         +--- com.azure:azure-core-tracing-opentelemetry -> 1.0.0-beta.36
|         |    \--- com.azure:azure-core:1.40.0
|         |         +--- com.azure:azure-json:1.0.1
|         +--- com.azure:azure-messaging-servicebus -> 7.14.1
|         |    +--- com.azure:azure-core:1.40.0 (*)
|         |    +--- com.azure:azure-core-amqp:2.8.6
|         |    |    +--- com.azure:azure-core:1.40.0 (*)
|         |    |    +--- com.microsoft.azure:qpid-proton-j-extensions:1.2.4
|         |    +--- com.azure:azure-core-http-netty:1.13.4
|         |    |    +--- com.azure:azure-core:1.40.0 (*)
|         +--- com.azure:azure-identity -> 1.9.1
|         |    +--- com.azure:azure-core:1.40.0 (*)
|         |    +--- com.azure:azure-core-http-netty:1.13.4 (*)
|         |    +--- com.microsoft.azure:msal4j:1.13.8
|         |    +--- com.microsoft.azure:msal4j-persistence-extension:1.2.0
|         |    |    +--- com.microsoft.azure:msal4j:1.4.0 -> 1.13.8 (*)
+--- dk.bankdata.qua:azure-webpubsub-tools:1.0.0
|    +--- com.azure:azure-core-tracing-opentelemetry:1.0.0-beta.36 (*)
|    +--- com.azure:azure-messaging-webpubsub:1.2.4 -> 1.2.5
|    |    +--- com.azure:azure-core:1.40.0 (*)
|    |    +--- com.azure:azure-core-http-netty:1.13.4 (*)
|    |    |    +--- com.azure:azure-messaging-servicebus:7.13.3 -> 7.14.1 (c)
|    |    |    +--- com.azure:azure-identity:1.8.1 -> 1.9.1 (c)
|    |    |    +--- com.azure:azure-core:1.37.0 -> 1.40.0 (c)
|    |    |    +--- com.azure:azure-core-amqp:2.8.3 -> 2.8.6 (c)
|    |    |    +--- com.microsoft.azure:msal4j:1.13.5 -> 1.13.8 (c)
|    |         |    +--- com.azure:azure-sdk-bom:1.2.14
|    |         |    |    +--- com.azure:azure-messaging-servicebus:7.14.1 (c)
|    |         |    |    +--- com.azure:azure-identity:1.9.1 (c)
|    |         |    |    +--- com.azure:azure-core:1.40.0 (c)
|    |         |    |    +--- com.azure:azure-core-amqp:2.8.6 (c)
|    |         |    |    +--- com.azure:azure-core-http-netty:1.13.4 (c)
|    |         |    |    \--- com.azure:azure-json:1.0.1 (c)
|    |         |    +--- com.azure:azure-core-tracing-opentelemetry:1.0.0-beta.36 (c)
|    |         |    \--- com.microsoft.azure:msal4j:1.13.8 (c)
|    |         +--- com.azure:azure-core-tracing-opentelemetry -> 1.0.0-beta.36
|    |         |    \--- com.azure:azure-core:1.40.0
|    |         |         +--- com.azure:azure-json:1.0.1
|    |         +--- com.azure:azure-messaging-servicebus -> 7.14.1
|    |         |    +--- com.azure:azure-core:1.40.0 (*)
|    |         |    +--- com.azure:azure-core-amqp:2.8.6
|    |         |    |    +--- com.azure:azure-core:1.40.0 (*)
|    |         |    |    +--- com.microsoft.azure:qpid-proton-j-extensions:1.2.4
|    |         |    +--- com.azure:azure-core-http-netty:1.13.4
|    |         |    |    +--- com.azure:azure-core:1.40.0 (*)
|    |         +--- com.azure:azure-identity -> 1.9.1
|    |         |    +--- com.azure:azure-core:1.40.0 (*)
|    |         |    +--- com.azure:azure-core-http-netty:1.13.4 (*)
|    |         |    +--- com.microsoft.azure:msal4j:1.13.8
|    |         |    +--- com.microsoft.azure:msal4j-persistence-extension:1.2.0
|    |         |    |    +--- com.microsoft.azure:msal4j:1.4.0 -> 1.13.8 (*)
\--- com.azure:azure-sdk-bom -> 1.2.14 (*)
|    \--- com.azure:azure-sdk-bom:1.2.14
\--- com.azure:azure-sdk-bom -> 1.2.14
|    +--- com.azure:azure-sdk-bom:1.2.14
|    |    +--- com.azure:azure-messaging-webpubsub:1.2.5 (c)
|    |    +--- com.azure:azure-core:1.40.0 (c)
|    |    +--- com.azure:azure-core-http-netty:1.13.4 (c)
|    |    +--- com.azure:azure-messaging-servicebus:7.14.1 (c)
|    |    +--- com.azure:azure-identity:1.9.1 (c)
|    |    +--- com.azure:azure-json:1.0.1 (c)
|    |    \--- com.azure:azure-core-amqp:2.8.6 (c)
|    +--- com.azure:azure-core-tracing-opentelemetry:1.0.0-beta.36 (c)
|    \--- com.microsoft.azure:msal4j:1.13.8 (c)
+--- com.azure:azure-sdk-bom -> 1.2.14 (*)
|    |    +--- com.azure:azure-core:1.37.0 -> 1.40.0 (c)
|    |    +--- com.azure:azure-messaging-servicebus:7.13.3 -> 7.14.1 (c)
|    |    +--- com.azure:azure-identity:1.8.1 -> 1.9.1 (c)
|    |    +--- com.azure:azure-core-amqp:2.8.3 -> 2.8.6 (c)
|    |    +--- com.microsoft.azure:msal4j:1.13.5 -> 1.13.8 (c)
|         +--- com.azure:azure-core-tracing-opentelemetry -> 1.0.0-beta.36
|         |    \--- com.azure:azure-core:1.40.0
|         |         +--- com.azure:azure-json:1.0.1
|         +--- com.azure:azure-messaging-servicebus -> 7.14.1
|         |    +--- com.azure:azure-core:1.40.0 (*)
|         |    +--- com.azure:azure-core-amqp:2.8.6
|         |    |    +--- com.azure:azure-core:1.40.0 (*)
|         |    |    +--- com.microsoft.azure:qpid-proton-j-extensions:1.2.4
|         |    +--- com.azure:azure-core-http-netty:1.13.4
|         |    |    +--- com.azure:azure-core:1.40.0 (*)
|         +--- com.azure:azure-identity -> 1.9.1
|         |    +--- com.azure:azure-core:1.40.0 (*)
|         |    +--- com.azure:azure-core-http-netty:1.13.4 (*)
|         |    +--- com.microsoft.azure:msal4j:1.13.8
|         |    +--- com.microsoft.azure:msal4j-persistence-extension:1.2.0
|         |    |    +--- com.microsoft.azure:msal4j:1.4.0 -> 1.13.8 (*)
+--- dk.bankdata.qua:azure-webpubsub-tools:1.0.0
|    +--- com.azure:azure-core-tracing-opentelemetry:1.0.0-beta.36 (*)
|    +--- com.azure:azure-messaging-webpubsub:1.2.4 -> 1.2.5
|    |    +--- com.azure:azure-core:1.40.0 (*)
|    |    +--- com.azure:azure-core-http-netty:1.13.4 (*)
|    +--- com.azure:azure-sdk-bom:1.2.14
+--- com.azure:azure-sdk-bom -> 1.2.14
+--- dk.bankdata.qua:azure-webpubsub-tools:1.0.0
|    +--- com.azure:azure-sdk-bom:1.2.14
|    |    +--- com.azure:azure-messaging-webpubsub:1.2.5 (c)
|    |    +--- com.azure:azure-core:1.40.0 (c)
|    |    +--- com.azure:azure-core-http-netty:1.13.4 (c)
|    |    +--- com.azure:azure-messaging-servicebus:7.14.1 (c)
|    |    +--- com.azure:azure-identity:1.9.1 (c)
|    |    +--- com.azure:azure-json:1.0.1 (c)
|    |    \--- com.azure:azure-core-amqp:2.8.6 (c)
|    +--- com.azure:azure-core-tracing-opentelemetry:1.0.0-beta.36 (c)
|    \--- com.microsoft.azure:msal4j:1.13.8 (c)
+--- com.azure:azure-sdk-bom -> 1.2.14 (*)
|    |    +--- com.azure:azure-core:1.37.0 -> 1.40.0 (c)
|    |    +--- com.azure:azure-messaging-servicebus:7.13.3 -> 7.14.1 (c)
|    |    +--- com.azure:azure-identity:1.8.1 -> 1.9.1 (c)
|    |    +--- com.azure:azure-core-amqp:2.8.3 -> 2.8.6 (c)
|    |    +--- com.microsoft.azure:msal4j:1.13.5 -> 1.13.8 (c)
|         +--- com.azure:azure-core-tracing-opentelemetry -> 1.0.0-beta.36
|         |    \--- com.azure:azure-core:1.40.0
|         |         +--- com.azure:azure-json:1.0.1
|         +--- com.azure:azure-messaging-servicebus -> 7.14.1
|         |    +--- com.azure:azure-core:1.40.0 (*)
|         |    +--- com.azure:azure-core-amqp:2.8.6
|         |    |    +--- com.azure:azure-core:1.40.0 (*)
|         |    |    +--- com.microsoft.azure:qpid-proton-j-extensions:1.2.4
|         |    +--- com.azure:azure-core-http-netty:1.13.4
|         |    |    +--- com.azure:azure-core:1.40.0 (*)
|         +--- com.azure:azure-identity -> 1.9.1
|         |    +--- com.azure:azure-core:1.40.0 (*)
|         |    +--- com.azure:azure-core-http-netty:1.13.4 (*)
|         |    +--- com.microsoft.azure:msal4j:1.13.8
|         |    +--- com.microsoft.azure:msal4j-persistence-extension:1.2.0
|         |    |    +--- com.microsoft.azure:msal4j:1.4.0 -> 1.13.8 (*)
+--- dk.bankdata.qua:azure-webpubsub-tools:1.0.0
|    +--- com.azure:azure-core-tracing-opentelemetry:1.0.0-beta.36 (*)
|    +--- com.azure:azure-messaging-webpubsub:1.2.4 -> 1.2.5
|    |    +--- com.azure:azure-core:1.40.0 (*)
|    |    +--- com.azure:azure-core-http-netty:1.13.4 (*)

@joshfree
Copy link
Member

joshfree commented Aug 2, 2023

Hi @AndreasPetersen, thank you for the detailed write up. @anuchandy will follow up here. Please note that if this is an urgent issue that we recommend creating an Azure Support ticket rather than a GitHub issue: https://github.com/Azure/azure-sdk-for-java/blob/main/SUPPORT.md#azure-support-tickets

@AndreasPetersen
Copy link
Author

Hi @joshfree , thanks for the reply. In fact we just created a support ticket a few days ago. I have already referred to this issue.

@anuchandy
Copy link
Member

Hello @AndreasPetersen, I'm closing this given that there is no more down time since you switched to the beta library and iiuc the application is receiving without interruption for ~1.5 months at this point.

@github-actions github-actions bot locked and limited conversation to collaborators Dec 26, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@anuchandy anuchandy

Labels
Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. needs-team-triage Workflow: This issue needs the team to triage. question The issue doesn't require a change to the product in order to be resolved. Most issues start as that Service Bus
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@joshfree @anuchandy @AndreasPetersen @JimSuplizio @azure-sdk