[improve][broker] Don't use forkjoin pool by default for deleting partitioned topics

[lhotari]

Motivation

While investigating some recent test failures and Pulsar CI OOME issues, #22588, the heap dumps revealed that
the ForkJoin pool's work queue is retaining a lot of CompletableFuture related instances.

The heap dump confirms that these are originating from NamespaceResources.PartitionedTopicResources#runWithMarkDeleteAsync method in this location:

pulsar/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/resources/NamespaceResources.java

Lines 345 to 374 in bbdc173

	markPartitionedTopicDeletedAsync(topic).whenCompleteAsync((markResult, markExc) -> {
	final boolean mdFound;
	if (markExc != null) {
	if (markExc.getCause() instanceof MetadataStoreException.NotFoundException) {
	mdFound = false;
	} else {
	log.error("Failed to mark the topic {} as deleted", topic, markExc);
	future.completeExceptionally(markExc);
	return;
	}
	} else {
	mdFound = true;
	}
	supplier.get().whenComplete((deleteResult, deleteExc) -> {
	if (deleteExc != null && mdFound) {
	unmarkPartitionedTopicDeletedAsync(topic)
	.thenRun(() -> future.completeExceptionally(deleteExc))
	.exceptionally(ex -> {
	log.warn("Failed to unmark the topic {} as deleted", topic, ex);
	future.completeExceptionally(deleteExc);
	return null;
	});
	} else if (deleteExc != null) {
	future.completeExceptionally(deleteExc);
	} else {
	future.complete(deleteResult);
	}
	});
	});

The assumption is that graceful shutdown in tests without causing memory leaks will be more achievable with the provided executor since the instance isn't shared like the Fork join common pool executor.

Modifications

Instead of using the forkjoin pool by default in NamespaceResources.PartitionedTopicResources#runWithMarkDeleteAsync, use a specific executor. For Pulsar broker, this would be the executor returned by PulsarService#getExecutor()

Documentation

• doc
• doc-required
• doc-not-needed
• doc-complete

[heesung-sn]

Looks good to me, but this could impact the performance. I would check with others if there are any performance concerns here.