Use graalVM native image to build client and client admin

[waxzce]

Using the pulsar-admin command can be very frustrating because of the startup time. That's one point were graalVM native image is interesting: build a static "binary".

Various problem to compile to native-image can occur, using reflection and dynamic classloader are often the problems providers. From my point of view, auth plugins are the main problem, but apart of these, do you see any points to look at?

[aahmed-se]

I don't know how stable graalVM is last I checked it's still is an RC candidate we might also need to look at other http client libs that support aot code generation.

[sijie]

alternatively, you can now use pulsarctl. it is a static linked CLI written in golang.

[fantapsody]

Pulsar itself should also be built with AOT compilers to reduce the time to start the process. Sometimes it takes up to a minute to start the broker with function worker enabled.

[harissecic]

I'm experiencing similar issues trying to make Micronaut integration for Pulsar Java client is really hard mainly due to reflection and Micronaut now demands that it can build for GraalVM native image.
If anyone has advice how to make this library run (which reflection calls to add) please let me know. I support @fantapsody as Java client library is a bit too heavy now to run due to many in-between calls which are mainly just using reflection to call other classes. This would maybe increase compile time but it's much more beneficial in the end to not have it or have as less as posible.

[aahmed-se]

Broker will require many changes and unlikely it can be made to work with graalvm anytime soon. Client should not an issue it has a dependency on netty , but netty authors should have already provided a guide to to use it with graal and it should work just fine.

[harissecic]

Broker will require many changes and unlikely it can be made to work with graalvm anytime soon. Client should not an issue it has a dependency on netty , but netty authors should have already provided a guide to to use it with graal and it should work just fine.

Shaded client which is the one I have to use in this case as netty needs to be supported independently of the Pulsar Client so I really can't due to other limitations extract it. I can only research again what netty methods are used with reflection and try to list them with org.apache.pulsar... prefix. On the other hand looking at the source code I'm not sure how safe calls to Schema instantiation would be which heavily rely on reflection. In any case I've made own Jackson schema to support passing in ObjectMapper from framework while using shaded library. In many cases this library tends to give me warnings and such running on new Java versions like 14. This is why it would be nice if possible to have less of those reflection and unsafe calls. On the other hand I guess main issue is in fact in netty in my case I just have to list all of those calls with apache pulsar extension as package name. An example from my point of view is

public static Schema<byte[]> newBytesSchema() {
        return catchExceptions(
                () -> (Schema<byte[]>) newClassInstance("org.apache.pulsar.client.impl.schema.BytesSchema")
                        .newInstance());
    }

    public static Schema<String> newStringSchema() {
        return catchExceptions(
                () -> (Schema<String>) newClassInstance("org.apache.pulsar.client.impl.schema.StringSchema")
                        .newInstance());
    }

In my case it's replaced by simple new StringSchema() calls instead of Schema<String> STRING = DefaultImplementation.newStringSchema();. I get that most of reflective calls that it uses are "safe" around graal but not really liked.
In any case thanks for the response and I'll try to figure out how to make this specific use case work because I expect it's not really that applicable (having to use shaded but having to list netty calls for graal yet again having to pass in your own jackson mapper).

[dave2wave]

The slow startup of pulsar-admin is why pulsar-shell was created.