Support for explicit HTTP parts

[bterlson]

We have made some initial forays into multipart/form-data by piggy-backing off of the OpenAPI approach, which essentially maps a JSON-like structure to parts through various rules, like the object keys are the form name, "bytes" means "files", arrays mean multiple parts, etc. This works well for simple cases, but leaves behind some gaps, and has some non-optimal behavior. In particular:

1. No way to override the content type of a specific part
2. No natural way to represent files with optional/required file name/content-type
3. Ambiguity between array-of-parts vs. part-that-is-array
4. No way to define custom headers for a particular part
5. No way to control the various serialization options like explode, deepObject, etc.
6. No way to explicitly configure a boundary (unsure if necessary)

Overall, while the current design of making MFD endpoints "feel like" they take an object, and emitters doing the best they can to represent an endpoint that takes that object across multiple parts, this leaves a lot of gaps.

OpenAPI addresses these limitations with the encoding property that lives alongside schema:

requestBody:
  content:
    multipart/form-data:
      schema:
        type: object
        properties:
          file: # a part
      encoding:
        file: #encoding for file

multipart/form-data is a more specialized form of multipart requests. OpenAPI has support for multipart/mixed, and uses the same definition for it (NB: it is not clear whether the schema properties are reflected in a multipart/mixed request in the same way they are for multipart/form-data using the content disposition header, but this would be strange given that multipart/mixed doesn't require usage of the content-disposition header).

This proposal aims to address these deficiencies.

Full example

model Person {
  fullName: string;
  address: string;
}

model File {};

@maxItems(1)
@minItems(3)
model FileArray is HttpPart<File, #{ name: "File" }>[];

model ImageFile is File {
  contentType: "image/jpeg" | "image/png"
}

model RequiredFileMetadata is File {
  contentType: string;
  filename: string;
}

op upload(
  @header `content-type`: "multipart/form-data",
  @multipartBody body: [
    // # String body
    HttpPart<string, #{ name: "fullName" }>,
    // # String body, name inferred from body:
    HttpPart<{ @body fullName: string }>,
    // # Object body
    HttpPart<Person, #{ name: "Person" }>,
    // # Array body (JSON array)
    HttpPart<Person[], #{ name: "People" }>,
    // # Multiple parts
    HttpPart<string, #{ name: "Names"}>[],
    // # Custom headers
    HttpPart<{@header custom: "hi!" , @body name: string }>,

    // # File using Bytes
    HttpPart<bytes, #{ name: "file" }>,
    // # File using File
    HttpPart<File, #{ name: "file" }>,
    // # Multiple files
    HttpPart<File, #{ name: "file" }>[],
    // # Between 1 and 3 files
    FileArray,
    // # Limited content types
    ImageFile,
    // # Limited content types, but inline
    HttpPart<{ @header `content-type`: "image/jpeg", @body image: File }>,
    // # With required metadata
    HttpPart<RequiredFileMetadata, #{ name: "file" }>,
    

    // # ERROR CASES
    // ## No name specified, required for multipart/form-data parts
    HttpPart<Person>
  ]
]): void;

This same spec would work also for multipart/mixed. However, for the case of multipart/form-data in particular, where every part is required to have a name, we have a convenience form that takes an object:

op upload(
  @header `content-type`: "multipart/form-data",
  @multipartBody body: {
    // # String body
    fullName: HttpPart<string>,
    headShots: HttpPart<Image>[]
  }
): void;

This form is very similar to what we have today, but the values must be wrapped in HttpPart. So for determining the name of a part, the following is the precedence order:

1. The explicitly provided name on the part
2. Name of the property in the object form
3. Inferred from body parameter

If none of these are specified, it's an error.

@multipartBody

This decorator is used in place of @body to signify a multipart request/response. It is an error for an HttpPart to appear in a regular @body. It is an error for a multipartBody to be anything except an array of HttpParts.

HttpPart<TBodyRoot extends {}, TOptions extends valueof PartOptions>

The HttpPart has two parameters, the body root of the part, and options. The body root behaves identically to the argument list of an operation. It should always be safe to refactor an operation into a multipart operation by copying the parameter list of the operation into an anonymous model passed to HttpPart.

It is an error for a multipart/form-data part to specify the content-disposition header, as this is essentially fixed for this content type.

File

The file type is a well-known type representing a File:

@Internal.httpFile
model File {
  contentType?: string;
  filename?: string;
  contents: bytes
}

Usage of file is not required when uploading file contents for multipart requests. When merely bytes is used, our existing behavior continues (which is essentially to treat bytes in a multipart/form-data request as if it were a File).

File exists for two reasons:

1. Allows changing the requiredness of the optional metadata properties
2. Allows uniform API descriptions across content types, e.g. when uploading a file over application/json, the File is just a regular object with three properties, but when uploading a file over a multipart request, emitters can know that the additional properties on the well-known File can be serialized in the content-disposition and content-type headers.

Internal @httpFile

Marks a model as an http file. Must have at least a contentType, filename, and contents property.

Deprecations

We should deprecate multipart/form-data with implicit or explicit body.

[MaryGao]

@timotheeguerin One more ask...
Recently we received the requirement of setting the default value for filename(see comment), currently JS implement this with customization code.

But I was wondering if we plan to accept this requirement in typespec side. Some service may want the default value for filename, some may not. Is it possible to allow service team to define this? maybe define in client-level setting or property-level one?

Also I notice that when the filename is optional, whether to set default value may be inconsistant cross language, so what is the final decision?

@timovv any insight?

[timotheeguerin]

I don't think this was part of this design, this only covers the fact that you can make the filename a required property in a part.(You can also set a default with a property default but that would be the service default) For client we don't have any defaults yet so feels like that design needs to be sorted out first.