Support multipart file part stream feeding

[pablolagreca]

Currently AHC support sending a multipart file part but reading from File or a byte array. It should be possible to also send a part which content is read from an InputStream.

For instance, I may be reading content from the network which I want to send as a multipart file part but I can't read the whole content into memory since te content is big.

[slandelle]

@pablolagreca Willing to contribute?

[wsargent]

It's early, but if you look at the WS docs involving iteratees, there may
be something there

https://www.playframework.com/documentation/2.3.x/ScalaWS

[wsargent]

Multipart form handling is something that WS itself is weak on, so
contributions welcome :-)

[slandelle]

Note that InputStream is a blocking API, so you might end up shooting yourself in the foot.
If the InputStream blocks, you'll block a Netty IO thread, meaning that not only the current channel/socket will be stalled, but also all the other ones associated with this thread.

[slandelle]

Should be quite easy to implement since #1018 refactoring.

Anyone interested, please give it a try (I won't myself).

[slandelle]

@pablolagreca Actually, what you want is not a java.io.InputStream which is a blocking crap.
What you want is something similar to FeedableBodyGenerator but for BodyPart.

[grossmueller-espirit]

Hi there,
is there any progress on this or is this feature still not supported?
Thanks for some information!

[slandelle]

Issue is still open and tagged with « contribution welcome » so....

[wsargent]

@grossmueller-espirit you can do this through play-ws using a stream:

https://github.com/playframework/play-ws#java

If you are looking for multipart formdata specifically, then that is a part of Play itself, because it relies on Play's understanding of MultipartFormData.

• https://www.playframework.com/documentation/2.6.x/ScalaWS#Submitting-multipart/form-data
• https://www.playframework.com/documentation/2.6.x/ScalaWS#Processing-large-responses

In practice, this means there's more library dependencies and creating the library is a bit different.

• https://www.playframework.com/documentation/2.6.x/ScalaWS#Directly-creating-WSClient

[samridh90]

@slandelle : I'm looking into implementing support for this. Can you elaborate a bit more on your comment?

We've been using AHC 2.3 and need to support streaming files uploads from an SFTP server to a cloud service. Ideally I'd like to avoid copying files to the local filesystem before uploading them to the cloud service which means I would use something like commons-vfs to get an input stream to the file on the remote SFTP target and just pipe that through to the other side.

If you can give me some more background on FeedableBodyGenerator and how you think piping InputStreams to multipart uploads should look I'll take a stab at implementing support for it in the next couple of days.

[slandelle]

I suspect there are some misunderstanding here.
What's laking is the way to pass a multipart part as an InputStream.

FeedableBodyGenerator is not for multipart, but for generating a full body (which is also possible directly with an InputStream).

If you intend to contribute, please have a look at https://github.com/AsyncHttpClient/async-http-client/tree/master/client/src/main/java/org/asynchttpclient/request/body/multipart/part.

[samridh90]

Thanks for the starting point @slandelle.
I was referring your comment further up in this thread:

@pablolagreca Actually, what you want is not a java.io.InputStream which is a blocking crap.
What you want is something similar to FeedableBodyGenerator but for BodyPart.

There seems to be some concern around InputStream being a blocking API and wanting to do something non-blocking like FeedableBodyGenerator. Is that still a valid concern or are we OK going down the InputStream route?