KAFKA-19998: KRaft client should expose the latest committed kraft version for some readers

[kevin-wu24]

Readers like the SimpleApiVersionsManager and the MetadataCache should only know about the latest committed kraft version. Others, like the broker registration manager, need to know the potentially uncommitted kraft version.

[chia7712]

@kevin-wu24 thanks for this patch. one dumb question: what are the drawbacks if API Versions response uses the "uncommitted" kraft version?

[kevin-wu24]

what are the drawbacks if API Versions response uses the "uncommitted" kraft version?

It's mainly a UX issue that impacts clusters undergoing a kraft version upgrade, or clusters who have not committed a "bootstrapped" kraft version 1 yet (i.e. because of network partitioning).

The issue of exposing the uncommitted kraft version in the APIVersions response is that the finalized features level for kraft may be different depending on the node to which I send APIVersionsRequest. It also is not really "finalized." Metadata version also has this issue, which is being addressed as part of: #21122