[fix][client] Make protobuf-java dependency optional in java client libraries

[Shawyeok]

Fixes #23469

Motivation

In the Pulsar Java client libraries, the protobuf-java library is only used in ProtobufSchema, making it suitable as an optional dependency. However, it is mistakenly included in the pulsar-client-admin and pulsar-client-all shaded jar and declared as required dependency of pulsar-client, pulsar-client-original and pulsar-client-admin-original, as noted by @merlimat in this discussion.

PR #23468 moved protobuf-java from a shaded JAR to a declared dependency in the pom.xml, which helped avoid conflicts between protobuf-java and application dependencies. Since users only need protobuf-java when using ProtobufSchema, it makes sense to declare it as an optional dependency. This change reduces the number of dependencies in the Pulsar client libraries.

Verifying this change

1. Execute the following Maven command:

mvn clean install -am -pl pulsar-client-all,pulsar-client-shaded,pulsar-client-admin-shaded -DskipTests

2. Verify that the protobuf-java dependency is marked optional in the installed pom.xml files, e.g.

<dependency>
  <groupId>com.google.protobuf</groupId>
  <artifactId>protobuf-java</artifactId>
  <version>3.25.5</version>
  <scope>compile</scope>
  <optional>true</optional>
</dependency>

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

• Dependencies (add or upgrade a dependency)
• The public API
• The schema
• The default values of configurations
• The threading model
• The binary protocol
• The REST endpoints
• The admin CLI options
• The metrics
• Anything that affects deployment

Documentation

• doc
• doc-required
• doc-not-needed
• doc-complete

Matching PR in forked repository

PR in forked repository: Shawyeok#19

[github-actions]

@Shawyeok Please add the following content to your PR description and select a checkbox:

- [ ] `doc` <!-- Your PR contains doc changes -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
- [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->

[lhotari]

LGTM. Thanks for addressing this. This could be a minor breaking change for some clients since they would now have to include the protobuf-java version that they prefer. We'll just have to add a release note that this is change is made. Since it's a minor breaking change and not many have upgraded to Pulsar 4.0.x, I think it makes sense to get this included in 4.0.x .

[lhotari]

Build failure.

https://github.com/apache/pulsar/actions/runs/11977798414/job/3
3566154951?pr=23632#step:11:1

Run src/check-binary-license.sh ./distribution/server/target/apache-pulsar--bin.tar.gz && src/check-binary-license.sh ./distribution/shell/target/apache-pulsar-shell--bin.tar.gz
protobuf-java-3.25.5.jar mentioned in LICENSE, but not bundled

I'd guess that it's the distribution/shell/src/assemble/LICENSE.bin.txt file which should be updated or protobuf-java should be included as a dependency for the shell. I wonder if it's needed there? If not, the LICENSE.bin.txt file should be updated by removing the entry.

[Shawyeok]

I'd guess that it's the distribution/shell/src/assemble/LICENSE.bin.txt file which should be updated or protobuf-java should be included as a dependency for the shell. I wonder if it's needed there? If not, the LICENSE.bin.txt file should be updated by removing the entry.

@lhotari

Yes, the protobuf-java dependency should be removed as well.

The protobuf-java dependency in the pulsar-common module was introduced in #17962. It seems unnecessary now, and removing it would be a cleaner approach. Please take a look.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 74.32%. Comparing base (bbc6224) to head (9295b64).
Report is 756 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #23632      +/-   ##
============================================
+ Coverage     73.57%   74.32%   +0.75%     
- Complexity    32624    34464    +1840     
============================================
  Files          1877     1944      +67     
  Lines        139502   147202    +7700     
  Branches      15299    16240     +941     
============================================
+ Hits         102638   109408    +6770     
- Misses        28908    29338     +430     
- Partials       7956     8456     +500     

Flag	Coverage Δ
inttests	27.20% <ø> (+2.62%)	⬆️
systests	24.32% <ø> (-0.01%)	⬇️
unittests	73.71% <ø> (+0.86%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

see 667 files with indirect coverage changes