Skip to content
This repository has been archived by the owner on Sep 2, 2024. It is now read-only.

[0.25] KafkaChannel to init offsets before dispatcher #913

Conversation

aliok
Copy link
Member

@aliok aliok commented Oct 1, 2021

Manual cherry pick of the same fix in 0.24: #886

Release Note

- 🐛  Fixed a bug that was causing Kafka Channel dispatcher to miss some events in case the offset for the consumer group was not initialized

Docs

aliok added 2 commits October 1, 2021 15:34
[0.24] KafkaChannel to init offsets before dispatcher (knative-extensions#886)
@knative-prow-robot knative-prow-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 1, 2021
@google-cla google-cla bot added the cla: yes Indicates the PR's author has signed the CLA. label Oct 1, 2021
@knative-prow-robot knative-prow-robot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Oct 1, 2021
@aliok aliok changed the title Kafka ch event loss 025 back [0.25] KafkaChannel to init offsets before dispatcher Oct 1, 2021
@knative-metrics-robot
Copy link

The following is the coverage report on the affected files.
Say /test pull-knative-sandbox-eventing-kafka-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/channel/consolidated/dispatcher/dispatcher.go 64.5% 66.1% 1.6
pkg/channel/consolidated/reconciler/controller/kafkachannel.go 54.8% 54.2% -0.6
pkg/common/consumer/consumer_factory.go 87.2% 84.8% -2.4
pkg/common/consumer/consumer_manager.go 96.9% 95.6% -1.2
pkg/common/consumer/consumergroup_offsets_checker.go Do not exist 4.0%
pkg/common/kafka/offset/offsets.go 81.1% 81.5% 0.4
pkg/source/adapter/adapter.go 52.1% 55.4% 3.3

@knative-prow-robot
Copy link
Contributor

@aliok: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-knative-sandbox-eventing-kafka-go-coverage eddac47 link false /test pull-knative-sandbox-eventing-kafka-go-coverage

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@codecov
Copy link

codecov bot commented Oct 1, 2021

Codecov Report

❗ No coverage uploaded for pull request base (release-0.25@e0bdb38). Click here to learn what that means.
The diff coverage is n/a.

Impacted file tree graph

@@               Coverage Diff               @@
##             release-0.25     #913   +/-   ##
===============================================
  Coverage                ?   74.06%           
===============================================
  Files                   ?      139           
  Lines                   ?     6212           
  Branches                ?        0           
===============================================
  Hits                    ?     4601           
  Misses                  ?     1382           
  Partials                ?      229           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e0bdb38...eddac47. Read the comment docs.

Copy link
Contributor

@matzew matzew left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve

@knative-prow-robot knative-prow-robot added the lgtm Indicates that a PR is ready to be merged. label Oct 1, 2021
@knative-prow-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: aliok, matzew

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@knative-prow-robot knative-prow-robot merged commit 813d807 into knative-extensions:release-0.25 Oct 1, 2021
@aliok aliok deleted the kafka-ch-event-loss-025-back branch October 1, 2021 19:25
@aliok
Copy link
Member Author

aliok commented Oct 1, 2021

/cherrypick release-0.26

I think there are conflicts but gonna try anyway

@knative-prow-robot
Copy link
Contributor

@aliok: #913 failed to apply on top of branch "release-0.26":

Applying: Cherry pick 3f2a9d7
Using index info to reconstruct a base tree...
M	go.mod
M	go.sum
M	pkg/channel/consolidated/reconciler/controller/kafkachannel.go
M	pkg/channel/consolidated/reconciler/controller/kafkachannel_test.go
M	pkg/channel/distributed/dispatcher/controller/kafkachannel.go
M	pkg/channel/distributed/dispatcher/controller/kafkachannel_test.go
M	pkg/common/consumer/consumer_manager_test.go
M	pkg/source/adapter/adapter.go
M	vendor/modules.txt
Falling back to patching base and 3-way merge...
Auto-merging vendor/modules.txt
CONFLICT (content): Merge conflict in vendor/modules.txt
Removing vendor/knative.dev/networking/pkg/prober/prober.go
Removing vendor/knative.dev/networking/pkg/prober/doc.go
Removing vendor/knative.dev/networking/LICENSE
Removing third_party/VENDOR-LICENSE/knative.dev/networking/pkg/prober/LICENSE
Auto-merging pkg/source/adapter/adapter.go
Auto-merging pkg/common/consumer/consumer_manager_test.go
Auto-merging pkg/channel/distributed/dispatcher/controller/kafkachannel_test.go
Auto-merging pkg/channel/distributed/dispatcher/controller/kafkachannel.go
Removing pkg/channel/consolidated/status/status_test.go
Removing pkg/channel/consolidated/status/status.go
Removing pkg/channel/consolidated/reconciler/controller/lister.go
Auto-merging pkg/channel/consolidated/reconciler/controller/kafkachannel_test.go
CONFLICT (content): Merge conflict in pkg/channel/consolidated/reconciler/controller/kafkachannel_test.go
Auto-merging pkg/channel/consolidated/reconciler/controller/kafkachannel.go
Removing pkg/channel/consolidated/dispatcher/subscription_endpoint_test.go
Removing pkg/channel/consolidated/dispatcher/subscription_endpoint.go
Auto-merging go.sum
CONFLICT (content): Merge conflict in go.sum
Auto-merging go.mod
CONFLICT (content): Merge conflict in go.mod
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0001 Cherry pick 3f2a9d7
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

In response to this:

/cherrypick release-0.26

I think there are conflicts but gonna try anyway

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

aliok added a commit to aliok/eventing-kafka that referenced this pull request Oct 1, 2021
aliok added a commit to aliok/eventing-kafka that referenced this pull request Oct 8, 2021
…ions#913)

* Cherry pick 3f2a9d7

[0.24] KafkaChannel to init offsets before dispatcher (knative-extensions#886)

* ./hack/update-codegen.sh
aliok added a commit to aliok/eventing-kafka that referenced this pull request Oct 8, 2021
…ions#913)

* Cherry pick 3f2a9d7

[0.24] KafkaChannel to init offsets before dispatcher (knative-extensions#886)

* ./hack/update-codegen.sh
aliok added a commit to aliok/eventing-kafka that referenced this pull request Oct 8, 2021
…ions#913)

* Cherry pick 3f2a9d7

[0.24] KafkaChannel to init offsets before dispatcher (knative-extensions#886)

* ./hack/update-codegen.sh
aliok added a commit to aliok/eventing-kafka that referenced this pull request Oct 8, 2021
…ions#913)

* Cherry pick 3f2a9d7

[0.24] KafkaChannel to init offsets before dispatcher (knative-extensions#886)

* ./hack/update-codegen.sh
openshift-merge-robot pushed a commit to openshift-knative/eventing-kafka that referenced this pull request Oct 11, 2021
* [0.25] KafkaChannel to init offsets before dispatcher  (knative-extensions#913)

* Cherry pick 3f2a9d7

[0.24] KafkaChannel to init offsets before dispatcher (knative-extensions#886)

* ./hack/update-codegen.sh

* [0.25] KafkaChannel dispatcher offset checking improvements (knative-extensions#929)

* Fix Kafka channel event loss during subscription becoming ready

* Make it look like knative-extensions#926
pierDipi added a commit to pierDipi/eventing-kafka that referenced this pull request Nov 15, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cla: yes Indicates the PR's author has signed the CLA. lgtm Indicates that a PR is ready to be merged. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants