Support for MultiPart forms

[glassfishrobot]

The spec currently provides no specific support for multpart/form. While it is possible of course to have a MBR for the format and some utility libraries provide this (e.g. Apache Clerezza jaxrs.utils) many implemententations provide a tighter integration with annotation supporting the fields of MultiPart Forms (jersey with @FormDataParam and CXF with @multipart) code taht relies on these extension is no longer portable.

The Jax-rs spec should standartize support for this.

Affected Versions

[2.0]

[glassfishrobot]

@glassfishrobot Commented
Reported by rebach

[glassfishrobot]

@glassfishrobot Commented
anthony_ve said:
XMLHttpRequest Level 2 adds a new FormData interface [1]. When doing an ajax request with a FormData object [2], the content-type must be multipart/form-data [3].
Another well-known use case of multipart/form-data is uploading files.

In my opinion, these 2 use cases, combined with the fact that JAX-RS implementations already support this (but each in their own, incompatible way), is more than enough reason to standardize this in JAX-RS in Java EE 8.

[1] https://developer.mozilla.org/en-US/docs/Web/API/FormData
[2] https://xhr.spec.whatwg.org/#the-send%28%29-method
[3] https://fetch.spec.whatwg.org/#concept-bodyinit-extract

[glassfishrobot]

@glassfishrobot Commented
This issue was imported from java.net JIRA JAX_RS_SPEC-413

[glassfishrobot]

• Issue Imported From: https://github.com/jax-rs/api/issues/418
• Original Issue Raised By:@glassfishrobot
• Original Issue Assigned To: @glassfishrobot

[struberg]

+1 for adding MIME Multipart support to the JAX-RS API as first class citizen.

we currently have to always defer to internal CXF functions to upload documents and files, etc.

[struberg]

@mkarg pleeeeassse take over :D

[mkarg]

@struberg Assigned to me. I'll try to drive it.

[mkarg]

@spericas @chkal @andymc12 Marc is very keen on getting this standardized, as it supposedly works in all JAX-RS implementations already. What do you think?

[spericas]

@mkarg I agree this a good idea, clearly incompatible extensions are not. However, I think this task may be larger than what it looks like at first if we want to provide full support for multipart/form-data (including nesting, etc.). Hopefully the API will look similar in spirit to the current one for application/x-www-form-urlencoded but with the additional support needed for multipart.

[andymc12]

+1 (to standardizing multipart support) In WebSphere, we end up wrapping the CXF APIs that Mark mentioned. I'd prefer that they were first class citizens instead.

[38leinaD]

+1 had to revert to using a raw servlet recently to implement upload functionality in a portable way.

[struberg]

Yes, @38leinaD that might still be needed in some cases.
Well, actually depending if we get streaming right ;)

If the implementation first needs to suck in all the request and only then can decide what to do, then it's not as good. Think about uploading a MRT scan or DVD image with multiple Gigabytes. This might easily exceed your RAM.
But then otoh I'd be perfectly fine if those edge cases would still require native Servlet work. We just have to make up our mind and think about it. There's always gonan be some trade offs and clear boundaries...

[chkal]

A fully agree that having multipart support in JAX-RS would be really useful. It is a very common use case and should be supported by the JAX-RS API out of the box.

[mkarg]

@struberg Seems we all want to get multipart, so maybe you like to propose an API?

[mkarg]

@spericas Can you please create a 2.2 milestone, so this issue can be correctly moved into 2.2 instead of icebox? Thanks.

[mkarg]

@struberg Whether or not streaming buffers in RAM is an implementation detail, so "we" (= the API guys) cannot influence that. Besides that, I wonder how you plan to access the headers of the second MRT scan without buffering all the bytes of the first MRT scan? I mean, you do have to put the bytes somewhere to position your input stream at the next part's header.

[chkal]

Seems we all want to get multipart, so maybe you like to propose an API?

@mkarg Perhaps we should first create some kind of document that summarizes how the different APIs of the major implementations look like. This would be a great foundation for working on a standard API.

[mkarg]

@chkal Why not. But instead of "we" and "document" I would propose that each vendor writes up his solution briefly in this issue, or in a Wiki page.

[spericas]

Here is the link to Jersey's user guide that talks about multipart support:

https://jersey.github.io/documentation/latest/media.html#multipart

[andymc12]

Sorry for the long delay, but here is how CXF handles multipart support:

http://cxf.apache.org/docs/jax-rs-multiparts.html

It looks to have a lot in common with Jersey's approach (CXF's @Multipart ~= Jersey's @FormDataParam, MultipartBody ~= MultiPart, Attachment ~= BodyPart).

[watarhu]

This is how resteasy resolves multipart:

https://docs.jboss.org/resteasy/docs/1.1.GA/userguide/html/Multipart.html

Comparison table (TBR):

Library	Jersey	CFX	Resteasy
Multipart param	@FormDataParam	@Multipart	@MultipartForm + @FormData + @PartType
Multipart body	MultiPart	MultipartBody	MultipartInput
Body Part	Attachment	BodyPart	InputPart

[ronsigal]

For the record, we've gotten past 1.1: https://docs.jboss.org/resteasy/docs/4.0.0.Final/userguide/html/ ;-)

…

On 4/9/19 12:25 PM, Albert Farré Figueras wrote: This is how resteasy resolves multipart: https://docs.jboss.org/resteasy/docs/1.1.GA/userguide/html/Multipart.html Comparison table (TBR): Library Jersey CFX Resteasy Multipart param ***@***.***| ***@***.***| ***@***.***| + ***@***.***| + ***@***.***| Multipart body |MultiPart| |MultipartBody| |MultipartInput| Body Part |Attachment| |BodyPart| |InputPart| — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#418 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAg383_A2Y8tjoyKrZudKIQCtoU_myIqks5vfL8DgaJpZM4SoYKB>.

[igbluz]

Hi, as a User of JAX-RS I am looking forward to have one day multipart support defined as standard and that vendors are able to do some good implementation so that I can use it in a easy way. It is a very common use case for us and I think for many in insurance and banking area and so should be supported by the JAX-RS API out of the box. So please continue with the effort to standardize for after getting some good solutions for OpenAPI generators and some support from OpenLiberty community or RedHat. Do you have a time line when it will be available?

[mkarg]

As "the big three" support multiform, it might be the time to discuss the adoption to our roadmap. Suggestions? :-)

[chkal]

+100

For me CDI alignment and multipart support are on the top of my priority list!

[mkarg]

Shall we add it to 2.3 or 3.0?

[andymc12]

Shall we add it to 2.3 or 3.0?

I'd say the sooner, the better. If it's too late to put into 2.2, then I'd vote for 2.3.

[igbluz]

+1 Great to see the work progress. Best regard, Igor

…

Am 09.08.2019 um 16:03 schrieb Andy McCright ***@***.***>: Shall we add it to 2.3 or 3.0? I'd say the sooner, the better. If it's too late to put into 2.2, then I'd vote for 2.3. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[mkarg]

@asoldano @jansupol +1 for adopting this in 2.3 already?

[igbluz]

Hi, is there any roadmap when version 2.3 should be available? I am still interested in the multipart feature to have it standardized. Where can I follow if some work is going on. Are there obstacles for continuing?

[asoldano]

@asoldano @jansupol +1 for adopting this in 2.3 already?

+1, sorry for the late reply (btw, please ping @ronsigal too in cases like this, as he leads the RESTEasy project with me).

[andymc12]

@igbluz the tentative roadmap is posted here:
https://github.com/eclipse-ee4j/jaxrs-api/wiki/Roadmap

The tentative release for 2.3 is 1Q2020.

[igbluz]

Thank you for answering so quickly, I look forward to 2020 so. Will be exciting to see the first results.

…

Am 02.10.2019 um 17:28 schrieb Andy McCright ***@***.***>:  @igbluz the tentative roadmap is posted here: https://github.com/eclipse-ee4j/jaxrs-api/wiki/Roadmap The tentative release for 2.3 is 1Q2020. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[erik-brangs]

Does this issue include support for multipart in the the JAX-RS client APIs (e.g. via Entity)?

[vanuatoo]

It's really strange that this is not part of the standard.
I would assume that any typical enterprise application has at least one feature that requires uploading files along with other data.
I think Jersey has a great support with custom annotations. RESTEasy is much harder and you have to do bean validation manually.
The lack of this feature in a standard breaks the interoperability goal of Jakarta EE.
Because of this we have to rewrite many parts of our application because we are moving from one server to another.
Please make this part of the standard.

[andymc12]

@mkarg - I've been doing a little bit of work in this space recently. If interested, I'd be willing to take it over - or work with you on it. Here is a rough idea of what I'd propose:

Add new APIs for Part (interface, probably in the ext package) and PartBuilder. Ideally, this would work with existing annotations. So let's say you have a service that produces and consumes multipart/form-data - it could look something like this:

@Path("/multi")
public class MyMultipartResource {

    @POST
    @Consumes(MediaType.MULTIPART_FORM_DATA)
    public Response postAsSingleEntity(List<Part> parts) {
        for (Part part : parts) {
            String name = part.getName(); // corresponds to the _name_ attribute in the Content-Disposition header
            String fileName = part.getFileName(); // could be Optional<String> ?? - corresponds to the _filename_ attr in C-D header
            InputStream is = part.getEntityStream();
            MultivaluedMap<String, String> partHeaders = part.getHeaders();
            MediaType mediaType = part.getMediaType();
            doSomethingWithPart(name, fileName, is, partHeaders, mediaType);
        }
        return Response.ok().build();
    }

    @POST
    @Consumes(MediaType.MULTIPART_FORM_DATA)
    @Path("asFormParams")
    public Response postAsFormParams(@FormParam("part1Name") String part1, 
                                     @FormParam("part2Name") InputStream part2,
                                     @FormParam("part3Name") Part part3) {
        // part1 is the string form of the entity stream for the part named "part1Name" ("name" attr in Content-Disposition header)
        // part2 is the entity stream of the part named "part2Name"
        // part3 is the full part - allowing the user to access headers, name, filename, etc.
        //...
    }

    @GET
    @Produces(MediaType.MULTIPART_FORM_DATA)
    public List<Part> getMultipartResponse() {
        List<Part> list = new ArrayList<>();
        list.add(PartBuilder.newBuilder(part1Name).fileName(part1FileName).entityStream(part1stream).build());
        list.add(PartBuilder.newBuilder(part2Name).headers(headerMap).mediaType(someMediaType).entityStream(part2Stream).build());
        //...
        return list;
    }
}

The idea would be that all of this code would work out of the box without the user requires to add any special entity providers. For the PartBuilder, the name and entity stream are required fields - all else (fileName, headers, mediaType - which defaults to application/octet-stream if a fileName is specified or text/plain if not) is optional.

We could also add a MultiPart entity class similar that could contain one or more Part instances, but I don't see much value in it other than as a wrapper around a collection of Parts - maybe somebody might want to specify the boundary parameter??

Wrt nesting, the multipart RFC (dated July 2015) mentions that the idea of using multiple nested files/parts under a "multipart/mixed" content-type is deprecated. Instead it suggests sending multiple parts with the same name, but different filename/content/etc. - so a user might do that like so:

Client c = ClientBuilder.newClient();
WebTarget target = c.target(someURL);
List<Part> parts = Arrays.asList(
    PartBuilder.newBuilder("name1").fileName("file1.doc").entityStream(stream1).build(),
    PartBuilder.newBuilder("name1").fileName("file2.doc").entityStream(stream2).build(),
    PartBuilder.newBuilder("name1").fileName("file3.doc").entityStream(stream3).build());
Entity entity = Entity.entity(parts, MediaType.MULTIPART_FORM_DATA);
Response r = target.request().post(entity);

I think if vendors want to support the deprecated approach, they are welcome to, but the spec should only require vendors to use the non-deprecated approach.

Any thoughts? I can draft up a pull request if that would be easier to digest. Thanks!

[paulrutter]

@andymc12

I have been using the latest 3.1 multipart API (List<EntityPart>) together with Jersey 3.1.5 (https://eclipse-ee4j.github.io/jersey.github.io/documentation/latest3x/media.html#multipart.server.rest).

I cannot seem to find any information on how to set limits for multipart form data uploads in https://jakarta.ee/specifications/restful-ws/3.1/jakarta-restful-ws-spec-3.1#consuming_multipart_formdata, like is possible with commons-fileupload (https://commons.apache.org/proper/commons-fileupload/using.html):

// Create a factory for disk-based file items
DiskFileItemFactory factory = new DiskFileItemFactory()
  // Set factory constraints
  .setSizeThreshold(yourMaxMemorySize)
  .setPath(yourTempDirectoryPath)
  .get();

// Create a new file upload handler
JakartaServletDiskFileUpload upload = new JakartaServletDiskFileUpload(factory);

// Set overall request size constraint
upload.setFileSizeMax(yourMaxRequestSize);

// Limit the number of parts
upload.setFileCountMax(20);

// Parse the request
List<DiskFileItem> items = upload.parseRequest(request);

Is this missing in the specification or should this be managed in the servlet container / manually in code?

[jansupol]

@paulrutter I can see Jersey is badly documented in this regard, but it supports some properties for the multipart. The MultipartProperties can be registered as ContextResolver<MultiPartProperties> for instance by

register(new MultiPartProperties().bufferThreshold(8192).tempDir("some-directory").resolver());

[paulrutter]

I checked it out, and this will work. But it misses several options to limit the number of parts being uploaded, and the limit per part. Is that something to file a feature request for in Jersey?

[jansupol]

Feel free to file it, we'll check how feasible is to deliver it. Please discuss what should happen in the case the data are over limits.