Skip to content
Pulsar CI

[improve][broker] Optimize high CPU usage when consuming from topics with ongoing txn #20764

Sign in to view logs

[improve][broker] Optimize high CPU usage when consuming from topics with ongoing txn

[improve][broker] Optimize high CPU usage when consuming from topics with ongoing txn #20764

Re-run triggered August 20, 2024 11:41
@coderzccoderzc
#23189
coderzc:improve_ongoing_txn_consume
Status Success
Total duration 45m 21s
Artifacts 13

pulsar-ci.yaml

on: pull_request
Preconditions
11s
Preconditions
Build and License check
10m 57s
Build and License check
Matrix: unit-tests
Build Pulsar docker image
21m 32s
Build Pulsar docker image
Matrix: Build Pulsar java-test-image docker image
CI - Unit - Upload Coverage
1m 17s
CI - Unit - Upload Coverage
Run CodeQL Analysis
19m 16s
Run CodeQL Analysis
Matrix: system-tests
Matrix: flaky-system-tests
Matrix: integration-tests
CI - System - Upload Coverage
3m 58s
CI - System - Upload Coverage
CI - Integration - Upload Coverage
2m 4s
CI - Integration - Upload Coverage
Build Pulsar on MacOS
8m 11s
Build Pulsar on MacOS
OWASP dependency check
0s
OWASP dependency check
Delete system test docker image artifact
11s
Delete system test docker image artifact
Delete integration test docker image artifact
11s
Delete integration test docker image artifact
Pulsar CI checks completed
11s
Pulsar CI checks completed
Fit to window
Zoom out
Zoom in

Annotations

46 warnings
CI - Unit - Other
Summary: Tests in class org.apache.pulsar.bookie.rackawareness.BookieRackAffinityMappingTest created 1 new threads. There are now 11 threads in total.
CI - Unit - Other
Summary: Tests in class org.apache.bookkeeper.mledger.impl.ManagedLedgerFactoryChangeLedgerPathTest created 16 new threads. There are now 26 threads in total.
CI - Unit - Other
Summary: Tests in class org.apache.bookkeeper.mledger.offload.filesystem.impl.FileSystemManagedLedgerOffloaderTest created 6 new threads. There are now 16 threads in total.
CI - Unit - Other
Summary: Tests in class org.apache.pulsar.testclient.PerfClientUtilsTest created 12 new threads. There are now 22 threads in total.
CI - Unit - Other
Summary: Tests in class org.apache.pulsar.testclient.Oauth2PerformanceTransactionTest created 3 new threads. There are now 13 threads in total.
CI - Unit - Other
Summary: Tests in class org.apache.pulsar.testclient.PerformanceProducerTest created 21 new threads. There are now 31 threads in total.
CI - Unit - Other
Summary: Tests in class org.apache.pulsar.testclient.PerformanceTransactionTest created 11 new threads. There are now 21 threads in total.
CI - Unit - Other
Summary: Tests in class org.apache.pulsar.broker.authentication.AuthenticationProviderAthenzTest created 1 new threads. There are now 11 threads in total.
CI - Unit - Other
Summary: Tests in class org.apache.pulsar.broker.authentication.ProxySaslAuthenticationTest created 2 new threads. There are now 12 threads in total.
CI - Unit - Other
Summary: Tests in class org.apache.pulsar.packages.management.storage.bookkeeper.BookKeeperPackagesStorageTest created 2 new threads. There are now 12 threads in total.
CI - Unit - Brokers - Broker Group 1
Summary: Tests in class org.apache.pulsar.PulsarBrokerStarterTest created 4 new threads. There are now 13 threads in total.
CI - Unit - Brokers - Broker Group 2
Summary: Tests in class org.apache.pulsar.broker.delayed.DelayedDeliveryTrackerFactoryTest created 1 new threads. There are now 10 threads in total.
CI - Unit - Brokers - Broker Group 2
Summary: Tests in class org.apache.pulsar.broker.zookeeper.ZKReconnectTest created 3 new threads. There are now 12 threads in total.
CI - Unit - Brokers - Broker Group 2
Summary: Tests in class org.apache.pulsar.broker.resourcegroup.ResourceGroupConfigListenerTest created 1 new threads. There are now 10 threads in total.
CI - Unit - Brokers - Broker Group 2
Summary: Tests in class org.apache.pulsar.broker.SameAuthParamsLookupAutoClusterFailoverTest created 2 new threads. There are now 12 threads in total.
CI - Unit - Brokers - Broker Group 2
Summary: Tests in class org.apache.pulsar.zookeeper.LocalBookkeeperEnsembleTest created 41 new threads. There are now 50 threads in total.
CI - Unit - Brokers - Client Impl
Summary: Tests in class org.apache.pulsar.client.impl.ReaderTest created 8 new threads. There are now 17 threads in total.
CI - Unit - Pulsar IO
Summary: Tests in class org.apache.pulsar.io.http.HttpSinkTest created 76 new threads. There are now 85 threads in total.
CI - Unit - Pulsar IO
Summary: Tests in class org.apache.pulsar.io.kinesis.KinesisSinkTest created 9 new threads. There are now 19 threads in total.
CI - Unit - Pulsar IO
Summary: Tests in class org.apache.pulsar.io.hdfs3.sink.text.HdfsStringSinkTests created 1 new threads. There are now 10 threads in total.
CI - Unit - Pulsar IO
Summary: Tests in class org.apache.pulsar.io.hdfs3.sink.seq.HdfsTextSinkTests created 1 new threads. There are now 10 threads in total.
CI - Unit - Pulsar IO
Summary: Tests in class org.apache.pulsar.io.hdfs3.sink.seq.HdfsSequentialSinkTests created 1 new threads. There are now 10 threads in total.
CI - Unit - Pulsar IO
Summary: Tests in class org.apache.pulsar.io.hdfs2.sink.text.HdfsStringSinkTests created 1 new threads. There are now 10 threads in total.
CI - Unit - Pulsar IO
Summary: Tests in class org.apache.pulsar.io.hdfs2.sink.seq.HdfsSequentialSinkTests created 1 new threads. There are now 10 threads in total.
CI - Unit - Pulsar IO
Summary: Tests in class org.apache.pulsar.io.hdfs2.sink.seq.HdfsTextSinkTests created 1 new threads. There are now 10 threads in total.
CI - Unit - Pulsar IO
Summary: Tests in class org.apache.pulsar.io.flume.node.TestPollingZooKeeperConfigurationProvider created 4 new threads. There are now 13 threads in total.
CI - Unit - Pulsar IO
Summary: Tests in class org.apache.pulsar.io.flume.source.StringSourceTests created 20 new threads. There are now 29 threads in total.
CI - Unit - Pulsar IO - Elastic Search
Summary: Tests in class org.apache.pulsar.io.elasticsearch.opensearch.OpenSearchClientTests created 2 new threads. There are now 11 threads in total.
CI - Unit - Pulsar IO - Elastic Search
Summary: Tests in class org.apache.pulsar.io.elasticsearch.opensearch.OpenSearchClientSslTests created 18 new threads. There are now 29 threads in total.
CI - Unit - Pulsar IO - Elastic Search
Summary: Tests in class org.apache.pulsar.io.elasticsearch.ElasticSearchBWCTests created 6 new threads. There are now 33 threads in total.
CI - Unit - Pulsar IO - Elastic Search
Summary: Tests in class org.apache.pulsar.io.elasticsearch.v8.ElasticSearch8AuthTests created 11 new threads. There are now 44 threads in total.
CI - Unit - Pulsar IO - Elastic Search
Summary: Tests in class org.apache.pulsar.io.elasticsearch.v8.ElasticSearch8SinkTests created 1 new threads. There are now 45 threads in total.
CI - Unit - Pulsar IO - Elastic Search
Summary: Tests in class org.apache.pulsar.io.elasticsearch.v8.ElasticSearch8SinkRawDataTests created 3 new threads. There are now 48 threads in total.
CI - Unit - Pulsar IO - Elastic Search
Summary: Tests in class org.apache.pulsar.io.elasticsearch.v8.ElasticSearch8ClientTests created 7 new threads. There are now 55 threads in total.
CI - Unit - Pulsar IO - Elastic Search
Summary: Tests in class org.apache.pulsar.io.elasticsearch.v7.ElasticSearch7ClientTests created 2 new threads. There are now 57 threads in total.
CI - Unit - Pulsar IO - Elastic Search
Summary: Tests in class org.apache.pulsar.io.elasticsearch.ElasticSearchRawRecordTests created 96 new threads. There are now 151 threads in total.
CI - Unit - Pulsar Metadata
Summary: Tests in class org.apache.pulsar.metadata.MetadataStoreTest created 8 new threads. There are now 18 threads in total.
CI - Unit - Pulsar Metadata
Summary: Tests in class org.apache.bookkeeper.replication.AuditorCheckAllLedgersTaskTest created 1 new threads. There are now 11 threads in total.
CI - Unit - Pulsar Metadata
Summary: Tests in class org.apache.bookkeeper.replication.AuditorRollingRestartTest created 47 new threads. There are now 57 threads in total.
CI - Unit - Pulsar Metadata
Summary: Tests in class org.apache.bookkeeper.replication.BookieAutoRecoveryTest created 14 new threads. There are now 24 threads in total.
CI - Unit - Pulsar Metadata
Summary: Tests in class org.apache.bookkeeper.replication.AuditorBookieTest created 3 new threads. There are now 13 threads in total.
CI - Unit - Pulsar Metadata
Summary: Tests in class org.apache.bookkeeper.replication.TestReplicationWorker created 49 new threads. There are now 59 threads in total.
CI - Unit - Pulsar Metadata
Summary: Tests in class org.apache.bookkeeper.replication.AutoRecoveryMainTest created 19 new threads. There are now 29 threads in total.
CI - Unit - Pulsar Metadata
Summary: Tests in class org.apache.bookkeeper.replication.AuditorPeriodicCheckTest created 11 new threads. There are now 21 threads in total.
CI - Unit - Brokers - Broker Group 3
Summary: Tests in class org.apache.pulsar.broker.admin.AdminApiTest created 1 new threads. There are now 10 threads in total.
Run CodeQL Analysis
1 issue was detected with this workflow: Please specify an on.push hook to analyze and see code scanning alerts from the default branch on the Security tab.

Artifacts

Produced during runtime
Name Size
Jacoco-coverage-report-unittests Expired
26.3 MB
Unit-BROKER_GROUP_3-dumps Expired
3.66 KB