Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add test coverage for improvement multipart encoded mode in rest-client #1853

Merged
merged 1 commit into from
Jun 27, 2024

Conversation

jcarranzan
Copy link
Contributor

@jcarranzan jcarranzan commented Jun 25, 2024

Summary

Adding TD for the next backport:
Improve the multipart encoded mode handling in the rest client

This PR adds test coverage for the improved multipart encoder mode handling in the REST client.
The tests cover the encoder modes (HTML5, RFC1738, RFC3986), verifying the correct encoding of multipart requests and successful parsing by a Vert.x server. Data integrity checks, including file content verification via size and headers, are performed.

Note: Direct verification of the specific encoder mode used during the transmission is not directly feasible, but
the tests ensure that the resulting content is correctly parsed and interpreted by the server, providing confidence that the appropriate encoder mode was applied.

Please select the relevant options.

  • Bug fix (non-breaking change which fixes an issue)
  • Dependency update
  • Refactoring
  • Backport
  • New scenario (non-breaking change which adds functionality)
  • This change requires a documentation update
  • This change requires execution against OCP (use run tests phrase in comment)

Checklist:

  • Methods and classes used in PR scenarios are meaningful
  • Commits are well encapsulated and follow the best practices

@jcarranzan jcarranzan force-pushed the simpler-encode-mode-test branch 5 times, most recently from ae1f07a to 4a471b5 Compare June 26, 2024 14:33
@github-actions github-actions bot added the triage/flaky-test Signal that flaky tests were detected during CI run label Jun 26, 2024
Copy link

Following jobs contain at least one flaky test: 'PR - Linux - JVM build - Latest Version'

@jcarranzan jcarranzan force-pushed the simpler-encode-mode-test branch 2 times, most recently from 81a27a8 to e0614c7 Compare June 26, 2024 16:01
@jcarranzan jcarranzan requested a review from rsvoboda June 26, 2024 16:02
@jcarranzan jcarranzan marked this pull request as ready for review June 26, 2024 16:02
Copy link

Following jobs contain at least one flaky test: 'PR - Linux - JVM build - Latest Version'

1 similar comment
Copy link

Following jobs contain at least one flaky test: 'PR - Linux - JVM build - Latest Version'

Copy link
Member

@rsvoboda rsvoboda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @jcarranzan.

I did the first quick check of this PR and there are some things I would like to have:

  • PR description doesn't add too much details about the test coverage, it points to the PR where I can get to the discussion, but that's 2 hops, short summary would make the life better for the reviewer
  • Short scenario summary should be present in HttpSimpleEncodeModeIT as JavaDoc, people from the team will be looking into this class sooner or later and having some context is better than needing to go through the commits history and reading the code
  • Having stuff in io.quarkus.ts.http.restclient.reactive is not intuitive as you are mixing new classes with the original ones, put new classes into io.quarkus.ts.http.restclient.reactive.multipart. Test can stay in the same package as it is now.
  • What's the motivation to use Vert.x server as the second server and not 2 Quarkus applications, just curious, not asking to rework stuff
  • I didn't notice any check to verify that the different encoderMode was used (not sure there is an easy way to do, but this is a thing that could be mentioned in PR description that it was considered), the test is checking that the correct content was received, but not how it was encoded ... in theory the multipart could be encoded using the same strategy all the time

@jcarranzan
Copy link
Contributor Author

Hi @jcarranzan.

I did the first quick check of this PR and there are some things I would like to have:

* PR description doesn't add too much details about the test coverage, it points to the PR where I can get to the discussion, but that's 2 hops, short summary would make the life better for the reviewer

* Short scenario summary should be present in HttpSimpleEncodeModeIT as JavaDoc, people from the team will be looking into this class sooner or later and having some context is better than needing to go through the commits history and reading the code

* Having stuff in `io.quarkus.ts.http.restclient.reactive` is not intuitive as you are mixing new classes with the original ones, put new classes into `io.quarkus.ts.http.restclient.reactive.multipart`. Test can stay in the same package as it is now.

* What's the motivation to use Vert.x server as the second server and not 2 Quarkus applications, just curious, not asking to rework stuff

* I didn't notice any check to verify that the different `encoderMode` was used (not sure there is an easy way to do, but this is a thing that could be mentioned in PR description that it was considered), the test is checking that the correct content was received, but not how it was encoded ... in theory the multipart could be encoded using the same strategy all the time

I agree, I will improve it with your suggestions, thanks @rsvoboda .

@michalvavrik
Copy link
Member

michalvavrik commented Jun 27, 2024

I didn't notice any check to verify that the different encoderMode was used ......

All that is true, but it's not possible to check this. Netty doesn't provide any API access to raw message that could give us a clue. Decoder result is always true. IMO:

@michalvavrik
Copy link
Member

hmm, I wonder if you could test this with Wiremock? Let's try that.

@jcarranzan jcarranzan force-pushed the simpler-encode-mode-test branch from 61f57cc to 06baa64 Compare June 27, 2024 10:18
@jcarranzan
Copy link
Contributor Author

jcarranzan commented Jun 27, 2024

I've improved the next steps:

  • PR description
  • JavaDoc in the test class
  • Rename of the test class

Answering some questions here:

  1. What's the motivation to use Vert.x server as the second server and not 2 Quarkus applications...

While initially, it seemed possible to verify the encoding mode directly using the Vert.x server as a lightweight option, further investigation didn't reveal a straightforward approach. I didn' consider about 2 Quarkus application, but yes could be other option.

  1. @rsvoboda about the last point "I didn't notice any check to verify that the different encoderMode was used (not sure there is an easy way to do, but this is a thing that could be mentioned in PR description that it was considered), the test is checking that the correct content was received, but not how it was encoded ... in theory the multipart could be encoded using the same strategy all the time"

It's true, directly inspecting the encoding mode in transit isn't straightforward, and I though checking the structure of the received multipart content was enough. I also tried the reproduced described here quarkusio/quarkus#39751 (reply in thread) but I noticed some things as server-undertow, etc that was out of scope of this testing.

PW: I am trying to figure out other ways to testing the encode mode as the Wiremock to see if it's possible at this moment...

@rsvoboda
Copy link
Member

What about proposed Wiremock approach? That could be a follow-up, not necessary to be in this PR.

Copy link
Member

@rsvoboda rsvoboda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is good enough.

I have 2 suggestions for the future improvement if you feel like it:

  • content of the uploaded files could be checked ... extend the DTO and store there buffer.toString()
  • add negative scenario - e.g. with incorrect multipart-post-encoder-mode

@jcarranzan
Copy link
Contributor Author

jcarranzan commented Jun 27, 2024

Ok now I've added those last 2 suggestions (check for the content of the uploaded files and the negative scenario). I will push them in some minutes.

@jcarranzan jcarranzan force-pushed the simpler-encode-mode-test branch from a7e2272 to 645be59 Compare June 27, 2024 13:26
@jcarranzan jcarranzan force-pushed the simpler-encode-mode-test branch from 645be59 to 650cd90 Compare June 27, 2024 13:34
Copy link

Following jobs contain at least one flaky test: 'PR - Linux - JVM build - Latest Version'

@rsvoboda rsvoboda merged commit f2cad72 into quarkus-qe:main Jun 27, 2024
12 checks passed
@michalvavrik michalvavrik removed the triage/flaky-test Signal that flaky tests were detected during CI run label Nov 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants