Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add test cases for newline handling in form payloads #26740

Merged
merged 15 commits into from
Apr 12, 2021
Merged

Add test cases for newline handling in form payloads #26740

merged 15 commits into from
Apr 12, 2021

Conversation

andreubotella
Copy link
Member

@andreubotella andreubotella commented Dec 3, 2020

(The title of this PR used to be "Add test cases for urlencoded serialization of filenames", dealing in particular with newline normalizations in the urlencoded serialization. However, the scope of the spec issues whatwg/url#562 and whatwg/html#6247 grew in scope as similar cases were found in other form enctypes.)

Copy link
Member

@annevk annevk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could make this a window.js test, but fine either way.

@andreubotella andreubotella requested a review from annevk December 3, 2020 12:32
Copy link
Member

@annevk annevk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I initially thought it was okay, but this is also testing form submission and that currently requires newline normalization for file names that the expected results do not seem to reflect.

And since this is really about form submission perhaps this should be placed in the html/ top-level directory, though I'm not entirely sure as there is no other way to test the encoding aspect of urlencoding... Either way we should probably link from a README on the other side, wherever we decide to put these.

@annevk
Copy link
Member

annevk commented Dec 4, 2020

Although this matches implementations per whatwg/url#562 (comment), I want to wait with landing these tests until the specifications back them up. Or am I misunderstanding something still?

@andreubotella
Copy link
Member Author

I wrote these tests are written to follow the specs, without the newline normalization, so that all browsers fail. Since we now have settled on a way to fix the spec to match the implementations, I'll update the tests to match.

@annevk
Copy link
Member

annevk commented Dec 4, 2020

Oh right, so much misunderstanding around these newlines. 😊

Are you also planning to test text/plain? Do you want help with the specification changes?

@andreubotella
Copy link
Member Author

Should I add the tests for text/plain to this PR?

@annevk
Copy link
Member

annevk commented Dec 4, 2020

Sounds great!

Copy link
Member

@annevk annevk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great! I can work on spec changes next week, but also happy to defer to you. Let me know.

@andreubotella
Copy link
Member Author

I think this is now ready for review. For any reviewers: note that the spec PR corresponding to these tests is whatwg/html#6287, and see also whatwg/html#6247 for some context.

@andreubotella andreubotella requested review from annevk and removed request for TakayoshiKochi January 14, 2021 16:02
@wpt-pr-bot wpt-pr-bot requested review from TakayoshiKochi and removed request for zqzhang January 15, 2021 10:59
moz-v2v-gh pushed a commit to mozilla/gecko-dev that referenced this pull request Jan 24, 2021
On `multipart/form-data` payloads, a null byte on the name, filename or string
value cuts off the rest of the name, filename or value. On `text/plain`
payloads, a null byte anywhere cuts off the rest of the entire payload.

This is because `nsLinebreakConverter::ConvertLineBreaks` is called without
giving a length parameter, which causes it to treat the input C string as
null-terminated.

The tests for `text/plain` are under review on WPT:
web-platform-tests/wpt#26740
(https://wpt.fyi/results/html/semantics/forms/form-submission-0/text-plain.window.html?label=pr_head&max-count=1&pr=26740)

Differential Revision: https://phabricator.services.mozilla.com/D102625
gecko-dev-updater pushed a commit to marco-c/gecko-dev-wordified that referenced this pull request Jan 25, 2021
On `multipart/form-data` payloads, a null byte on the name, filename or string
value cuts off the rest of the name, filename or value. On `text/plain`
payloads, a null byte anywhere cuts off the rest of the entire payload.

This is because `nsLinebreakConverter::ConvertLineBreaks` is called without
giving a length parameter, which causes it to treat the input C string as
null-terminated.

The tests for `text/plain` are under review on WPT:
web-platform-tests/wpt#26740
(https://wpt.fyi/results/html/semantics/forms/form-submission-0/text-plain.window.html?label=pr_head&max-count=1&pr=26740)

Differential Revision: https://phabricator.services.mozilla.com/D102625

UltraBlame original commit: 6c7f4b2ece67c32ae549c2d53efaf10e188ae182
jamienicol pushed a commit to jamienicol/gecko that referenced this pull request Jan 29, 2021
On `multipart/form-data` payloads, a null byte on the name, filename or string
value cuts off the rest of the name, filename or value. On `text/plain`
payloads, a null byte anywhere cuts off the rest of the entire payload.

This is because `nsLinebreakConverter::ConvertLineBreaks` is called without
giving a length parameter, which causes it to treat the input C string as
null-terminated.

The tests for `text/plain` are under review on WPT:
web-platform-tests/wpt#26740
(https://wpt.fyi/results/html/semantics/forms/form-submission-0/text-plain.window.html?label=pr_head&max-count=1&pr=26740)

Differential Revision: https://phabricator.services.mozilla.com/D102625
@andreubotella
Copy link
Member Author

Since it seems like implementations agree on the spec change, we should work towards getting this merged. Since this PR touched many files all over the place, there are many suggested reviewers, but in addition to @annevk, I'd like to get @tkent-google's approval, since in #26838 I changed the tests for form-associated custom elements to match the spec, not yet realizing that both Chrome and Firefox normalized strings added in the formdata event in the same way (and Safari only didn't because it doesn't implement that event). This PR changes that back and also adds tests for files in FACEs.

Copy link
Member

@annevk annevk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy with these tests. Thanks so much for taking the time to write them all! One thing that would have made reviewing slightly easier is if the function was defined before all the invocations to it, but it doesn't matter much.

@andreubotella
Copy link
Member Author

@tkent-google Ping

Copy link
Contributor

@tkent-google tkent-google left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@annevk annevk merged commit 9353d55 into web-platform-tests:master Apr 12, 2021
annevk pushed a commit to whatwg/html that referenced this pull request May 20, 2021
User agents normalize newlines when serializing form data to text/plain, application/x-www-form-urlencoded, and multipart/form-data. (This can be observed through FormData or the formdata event.)

This additionally changes the input passed to the application/x-www-form-urlencoded and text/plain serializers to be a
list of name-value pairs, where the values are always strings rather than potentially File objects.

Tests: web-platform-tests/wpt#26740.

Follow-up: #6624 & #6697.

Closes #6247. Helps with whatwg/url#562.
@andreubotella andreubotella deleted the urlencoded-serialization branch May 31, 2021 17:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants