Additional consumer functionality exposed

[dmedser]

New methods added to trait KafkaOffsets[F[_]]:

def committed(partitions: Set[TopicPartition]): F[Map[TopicPartition, OffsetAndMetadata]]

def committed(partitions: Set[TopicPartition], timeout: FiniteDuration): F[Map[TopicPartition, OffsetAndMetadata]]

[dmedser]

@bplommer @LMnet could you please give me a hint how to fix this CI error:

* abstract method committed(scala.collection.immutable.Set)java.lang.Object in interface fs2.kafka.consumer.KafkaOffsets is present only in current version?

Can I just add

ProblemFilters.exclude[ReversedMissingMethodProblem]("fs2.kafka.consumer.KafkaOffsets.committed")

to mimaBinaryIssueFilters?

[LMnet]

Hi @dmedser

Yes, you have to change MIMA rules to fix CI error.

[bplommer]

This is going to pass CI, but the change is genuinely going to break binary compatibility. So the options are either to retarget this PR at Series/3.x or add a new trait (KafkaOffsetsV2?) that extends KafkaOffsets.

[dmedser]

This is going to pass CI, but the change is genuinely going to break binary compatibility. So the options are either to retarget this PR at Series/3.x or add a new trait (KafkaOffsetsV2?) that extends KafkaOffsets.

@bplommer Does this mean that whenever someone wants to add some additional functionality, they will have to add a new trait with name KafkaXVN where X is Offsets and N is 2 as in my case?

Also if we consider this case with additional trait, what should KafkaConsumer look like? I suppose something like this:

sealed abstract class KafkaConsumer[F[_], K, V]
  extends KafkaConsume[F, K, V]
    with . . .
    with KafkaOffsetsV2[F]
    with . . .

where KafkaOffsetsV2 extends KafkaOffsets, right?

[LMnet]

change is genuinely going to break binary compatibility

Are you sure? As far as I know, adding new methods will not violate binary compatibility.

[bplommer]

Are you sure? As far as I know, adding new methods will not violate binary compatibility.

Yes, because it's an unsealed trait. A mock KafkaOffsets implementation compiled against a previous version won't have implementations, but a user of that mock compiled against the new version will expect them, leading to a possible NoSuchMethodException.

[bplommer]

@bplommer Does this mean that whenever someone wants to add some additional functionality, they will have to add a new trait with name KafkaXVN where X is Offsets and N is 2 as in my case?

Also if we consider this case with additional trait, what should KafkaConsumer look like? I suppose something like this:

sealed abstract class KafkaConsumer[F[_], K, V]
  extends KafkaConsume[F, K, V]
    with . . .
    with KafkaOffsetsV2[F]
    with . . .

where KafkaOffsetsV2 extends KafkaOffsets, right?

Yes, that's all correct.

[dmedser]

@bplommer Is everything correct this time?

[dmedser]

@bplommer @LMnet could you guys trigger pipeline again please?

[bplommer]

LGTM