Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doing base64 encode in channel's responseBody #6542

Conversation

liuchangyan
Copy link
Contributor

@liuchangyan liuchangyan commented Sep 27, 2022

Fixes #6547
Signed-off-by: Teresaliu changyan.liu@intel.com

Proposed Changes

  • Avoiding "invalid" HTTP header values for CE extension to encode channel's response body
  • Update unit and e2e tests
  • Add corresponding rekt test

Pre-review Checklist

  • At least 80% unit test coverage
  • E2E tests for any new behavior
  • Docs PR for any user-facing impact
  • Spec PR for any new API feature
  • Conformance test for any change to the spec

Release Note


Docs

@knative-prow knative-prow bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Sep 27, 2022
@codecov
Copy link

codecov bot commented Sep 27, 2022

Codecov Report

Base: 81.94% // Head: 81.89% // Decreases project coverage by -0.05% ⚠️

Coverage data is based on head (0719930) compared to base (7097458).
Patch coverage: 75.00% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6542      +/-   ##
==========================================
- Coverage   81.94%   81.89%   -0.06%     
==========================================
  Files         235      235              
  Lines       11738    11756      +18     
==========================================
+ Hits         9619     9627       +8     
- Misses       1646     1652       +6     
- Partials      473      477       +4     
Impacted Files Coverage Δ
pkg/channel/message_dispatcher.go 80.73% <75.00%> (-0.43%) ⬇️
pkg/eventfilter/subscriptionsapi/exact_filter.go 57.14% <0.00%> (-9.53%) ⬇️
pkg/eventfilter/subscriptionsapi/prefix_filter.go 68.96% <0.00%> (-7.04%) ⬇️
pkg/eventfilter/subscriptionsapi/suffix_filter.go 68.96% <0.00%> (-7.04%) ⬇️
pkg/reconciler/sugar/resources/broker.go 100.00% <0.00%> (ø)
pkg/broker/filter/filter_handler.go 75.85% <0.00%> (+0.42%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@matzew
Copy link
Member

matzew commented Sep 27, 2022

Could I suggest to also create an issue for this? It is at least related to the issue fixed here: knative-extensions/eventing-kafka-broker#2604

Also, it would be nice to have some rekt-test for this, again like in here: https://github.com/knative-sandbox/eventing-kafka-broker/pull/2604/files#diff-0de2d5afb422b7e648e6a13010e4331ecad2c15de240dcea0e8177e263e73e10

@liuchangyan
Copy link
Contributor Author

liuchangyan commented Sep 27, 2022

Could I suggest to also create an issue for this? It is at least related to the issue fixed here: knative-sandbox/eventing-kafka-broker#2604

Sure, I will open an issue and add some rekt-test for this PR

@@ -235,15 +236,20 @@ func (d *MessageDispatcherImpl) executeRequest(ctx context.Context,
execInfo.Time = dispatchTime

body := make([]byte, attributes.KnativeErrorDataExtensionMaxLength)
readLen, err := response.Body.Read([]byte(body))

// encodeBase64 encodes response body as base64 for the resulting length.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think if to long, we reduce it to first 1024

also this might need some reflection on some of the rekt/e2e tests (see failures)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it

@liuchangyan liuchangyan changed the title [WIP] Doing base64 encode in channel's responseBody Doing base64 encode in channel's responseBody Oct 14, 2022
@knative-prow knative-prow bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. area/test-and-release Test infrastructure, tests or release and removed do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Oct 14, 2022
@liuchangyan liuchangyan force-pushed the KNATIVE_base64_encode_data_in_executeRequest_channel_response_body branch from 1a933dd to a258baf Compare October 18, 2022 09:32
@knative-prow knative-prow bot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Oct 18, 2022
@knative-prow knative-prow bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Oct 18, 2022
@liuchangyan
Copy link
Contributor Author

/retest

Copy link
Member

@pierDipi pierDipi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

/cc @matzew

Comment on lines +305 to +321
func TestChannelSubscriptionReturnedErrorData(t *testing.T) {
t.Parallel()

ctx, env := global.Environment(
knative.WithKnativeNamespace(system.Namespace()),
knative.WithLoggingConfig,
knative.WithTracingConfig,
k8s.WithEventListener,
environment.Managed(t),
)

createSubscriberFn := func(ref *duckv1.KReference, uri string) manifest.CfgFn {
return subscription.WithSubscriber(ref, uri)
}

env.Test(ctx, t, channel.ChannelSubscriptionReturnedErrorData(createSubscriberFn))
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we additionally add a similar test that used the Broker/Trigger API?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The extensions is not added in Broker/Trigger, I am working on this PR #6541

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I missed your comment #6541 (comment), thanks!

@knative-prow knative-prow bot requested a review from matzew October 19, 2022 08:31
@knative-prow knative-prow bot added the lgtm Indicates that a PR is ready to be merged. label Oct 19, 2022
@pierDipi
Copy link
Member

/approve

@knative-prow
Copy link

knative-prow bot commented Oct 19, 2022

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: liuchangyan, pierDipi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@knative-prow knative-prow bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 19, 2022
@knative-prow knative-prow bot merged commit 98dfb5b into knative:main Oct 19, 2022
pierDipi added a commit to pierDipi/eventing that referenced this pull request May 2, 2023
Sanitizing the HTTP body was added as a measure to try to handle
as many HTTP body as possible, now that we're base64 encoding the
error body [1], it is not necessary anymore.

[1] knative#6542

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>
pierDipi added a commit to pierDipi/eventing that referenced this pull request May 2, 2023
Sanitizing the HTTP body was added as a measure to try to handle
as many HTTP body as possible, now that we're base64 encoding the
error body [1], it is not necessary anymore.

[1] knative#6542

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>
pierDipi added a commit to pierDipi/eventing that referenced this pull request May 2, 2023
Sanitizing the HTTP body was added as a measure to try to handle
as many HTTP body as possible, now that we're base64 encoding the
error body [1], it is not necessary anymore.

[1] knative#6542

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>
knative-prow bot pushed a commit that referenced this pull request May 2, 2023
Sanitizing the HTTP body was added as a measure to try to handle as many
HTTP body as possible, now that we're base64 encoding the error body
[1], it is not necessary anymore, it just breaks any consumer trying to
use such info.

[1] #6542

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>
pierDipi added a commit to pierDipi/eventing that referenced this pull request May 4, 2023
)

Sanitizing the HTTP body was added as a measure to try to handle as many
HTTP body as possible, now that we're base64 encoding the error body
[1], it is not necessary anymore, it just breaks any consumer trying to
use such info.

[1] knative#6542

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>
openshift-merge-robot pushed a commit to openshift-knative/eventing that referenced this pull request May 5, 2023
) (#203)

Sanitizing the HTTP body was added as a measure to try to handle as many
HTTP body as possible, now that we're base64 encoding the error body
[1], it is not necessary anymore, it just breaks any consumer trying to
use such info.

[1] knative#6542

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>
openshift-cherrypick-robot pushed a commit to openshift-cherrypick-robot/knative-eventing that referenced this pull request May 5, 2023
)

Sanitizing the HTTP body was added as a measure to try to handle as many
HTTP body as possible, now that we're base64 encoding the error body
[1], it is not necessary anymore, it just breaks any consumer trying to
use such info.

[1] knative#6542

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>
openshift-merge-robot pushed a commit to openshift-knative/eventing that referenced this pull request May 5, 2023
) (#211)

Sanitizing the HTTP body was added as a measure to try to handle as many
HTTP body as possible, now that we're base64 encoding the error body
[1], it is not necessary anymore, it just breaks any consumer trying to
use such info.

[1] knative#6542

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>
Co-authored-by: Pierangelo Di Pilato <pierdipi@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/test-and-release Test infrastructure, tests or release lgtm Indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Channel dispatcher error when response body has CRLF
3 participants