allow extra leading new lines in response

[LeiYangGH]

this library is great!
when i testing against my server, the library cannot detect the boundary.
after some debugging, i noticed our server adds a extra new line in response.
i changed the source a little and can perfectly run:

        private static string DetectBoundary(RebufferableBinaryReader reader)
        {
            // Presumably the boundary is --|||||||||||||| where -- is the stuff added on to
            // the front as per the protocol and ||||||||||||| is the part we care about.
            string line = null;
            while (string.IsNullOrWhiteSpace(line))
            {
                line = reader.ReadLine();
            }
//...
       }

[Jericho]

Interesting. I don't think I have ever heard of web servers adding content to a response. Luckily, in the case you are presenting, the inserted content is a blank line but, still, it doesn't seem right that a web server would take the liberty of inserting any content at all.

Do you know why your web server behaves this way?

[Jericho]

oh and also here's a quick workaround for you: you can bypass the boundary detection logic by specifying the boundary. In other words, rather than var parser = MultipartFormDataParser.Parse(stream);,

you can use var parser = MultipartFormDataParser.Parse(stream, "... your boundary marker ...");

[LeiYangGH]

i know it. but i like the boundary detection better.

[Bian-Sh]

我是 Unity 开发，我发现UnityWebRequest 配套的 MultipartFormFileSection 在拼接数据时就发现有空行。
从源码中就能看到的：
https://github.com/Unity-Technologies/UnityCsReference/blob/61f92bd79ae862c4465d35270f9d1d57befd1761/Modules/UnityWebRequest/Public/WebRequestExtensions.cs#LL372C41-L372C41

[Jericho]

@Bian-Sh Thank you for sharing this information. The code you shared shows that Unity's UnityWebRequest.Post adds a blank line before each section which will result in the same problem when you attempt to parse the resulting request with this library.

This is definitely an argument in favor of changing the behavior of the parse to ignore blank lines before the boundary marker rather than the current behavior which is to throw an exception.

@LeiYangGH Are you using Unity as well? Can you describe how your request are constructed so we can maybe understand why you get extra blank lines in the request payload.

[LeiYangGH]

@Jericho thanks for the attention! I'm actually looking for java parser but coincidently came across this library and finds it really facinating. My server is a tts engine that has been used for many years by our customers, I don't know the background why there's an extra newline. I just want the library be more robust(or tolerant of trivial mis alignment with rfc standards).

[Bian-Sh]

@Bian-Sh Thank you for sharing this information. The code you shared shows that Unity's UnityWebRequest.Post adds a blank line before each section which will result in the same problem when you attempt to parse the resulting request with this library.

This is definitely an argument in favor of changing the behavior of the parse to ignore blank lines before the boundary marker rather than the current behavior which is to throw an exception.

@LeiYangGH Are you using Unity as well? Can you describe how your request are constructed so we can maybe understand why you get extra blank lines in the request payload.

可以从 WebRequestExtensions.cs 366 - 372 行的这个逻辑 可以看到，这里存在一个多余的换行符导致了在 Headers 和 Datas之间形成了一个空行，这个空行导致检测Boundary失败：

                string contentType = section.contentType;
                if (!string.IsNullOrEmpty(contentType))
                {
                    header += "Content-Type: " + contentType + "\r\n";
                }

                formData.AddRange(crlf);

另外，@LeiYangGH 只修改了同步版本的函数 DetectBoundary，没有修改异步版本的 DetectBoundaryAsync

[Jericho]

可以从 WebRequestExtensions.cs 366 - 372 行的这个逻辑 可以看到，这里存在一个多余的换行符导致了在 Headers 和 Datas之间形成了一个空行，这个空行导致检测Boundary失败：

Yes, I got it. formData.AddRange(crlf); is indeed the culprit. Adding blank lines between sections is fine except before the first section because we currently assume that the boundary marker is on the first line as mentioned here:

https://github.com/Http-Multipart-Data-Parser/Http-Multipart-Data-Parser/blob/19fa86c659b70450d11ce2d83754efbe074b2834/Source/HttpMultipartParser/StreamingBinaryMultipartFormDataParser.cs#L242-L246

I will change this assumption to ignore blank lines in the upcoming 8.2.0 release.