Support for arbitrary query strings

[pofallon]

Hello! I'm trying to use the OpenAPI 3.0 spec to document an API that supports a subset of the Resource Query Language (RQL). However, because this query string format doesn't adhere to the typical name=value parameter structure, I'm struggling to express it in OpenAPI.

For example, a valid query string might be: http://localhost:5000/books?select(title,author)&gt(published,2000-06-30T12:20:08Z)&sort(published).

I wouldn't expect OpenAPI to understand RQL, but if there were a way to specify that an endpoint may have a query string, and that the query string contents are beyond the scope of OpenAPI, that would suffice. (I'm really thinking from the Swagger UI perspective -- so a user wanting to try the API can type in the entire RQL query string themselves.)

If anyone has any thoughts on other ways this can be accomplished with the existing 3.0 specification, that would be great too!

Thanks,
Paul

[handrews]

Hmm.... is there (or could there be) a media type for RQL query strings?

Technically, the name=value parameter structure is not a URI thing, it's the application/x-www-form-urlencoded media type. As far as URIs are concerned everything in between the ? and the # is the query string with no mandatory structure.

OpenAPI could offer an in value of queryContent or something like that, and then use the content field for that parameter to express RQL as a media type. Maybe? Just brainstorming a bit here, there might be obvious problems with this approach as I've not thought it through. I definitely have no idea what tooling challenges this might introduce (Hi @webron!)

(interestingly, this would also allow expressing relationships among query parameters as you could use queryContent with application/x-www-form-urlencoded and then apply schema to that to produce the name=value stuff - see also #256)

(also, I can't tell if #1054 is similar to this or not)

[hkosova]

As @handrews mentioned, you probably need to use the content keyword for your query parameter. content uses the media type identifier to define the serialization format for the parameter schema.

You can find a similar example for SPARQL at https://github.com/OAI/Gluecon-Workshop/blob/master/Different/parameters.md#complex-parameters-in-v3

In your case, you can try something like this:

    parameters:
    - name: complexQuery
      in: query
      content:
        application/rdf:
          schema:
            ???   # type: string? type: object?
          example: select(title,author)&gt(published,2000-06-30T12:20:08Z)&sort(published)

application/rql is the proposed media type for RDF according to http://dundalek.com/rql/draft-zyp-rql-00.html#rfc.section.13

[pofallon]

Thanks for the replies! I tried the content suggestion above and unfortunately it's putting [parameters.name]= in front of the RQL string. When I include allowReserved=true (and set the schema type to string or object) and supply a value of "select(author)&sort(published)" for this field in the Swagger Editor, it transforms into http://localhost:5000/books?complexQuery=select(author)&sort(published).

If there's a way to prevent it from adding complexQuery=, I'll be all set!

[handrews]

@pofallon yeah that is why I was thinking we'd need a new "in" value. in: query assumes application/x-www-form-urlencoded. The content just applies to the value of the URL-encoded value, not the query string as a whole.

So in: queryContent or in: wholeQueryString or something like that.

(troll mode: or, just fully support RFC 6570 :-)

[MikeRalphson]

If there's a way to prevent it from adding complexQuery=, I'll be all set!

@pofallon assuming setting the name of the parameter to '' (the empty string) doesn't work?

[pofallon]

That's a good suggestion! Unfortunately it appears that with either an empty name, or no name defined at all, it omits the parameter altogether.

[darrelmiller]

I knew we were going to get bitten by something like this sooner or later. As far as I know there is no official way to do this. I suspect the best way to allow this to happen would be to allow the simple style for parameters in query. The only question is how to delimit multiple simple parameters in the query string. I'm assuming using a comma would make the most sense. I'm not exactly sure how simple and form parameters in the query would interact.

[handrews]

The only question is how to delimit multiple simple parameters in the query string

Would it be reasonable to restrict the query string to a single parameter when using the simple style? Usually the rare occasions that I see the whole query string used as a string, there's some very specific syntax (such as the example given here) that's in use and won't mix well with anything else anyway.

It would be interesting if we could find any examples of multiple values in a free-form string that are not following an externally-defined syntax.

I'm not exactly sure how simple and form parameters in the query would interact.

My intuitive guess would be that when using form parameters you always have to end up with a valid application/x-www-form-urlencoded string in the query, which is part of why I'd assume that you couldn't mix these.

[darrelmiller]

@handrews Yeah, it would work to say style=simple can only be used in the query if it is the only query parameter.... but yuck. How many more of these ugly exceptions are we going to end up with? Makes me think the old adage is appropriate: URI templates are an icky solution that are just a little less icky than all the other solutions.

[approxit]

Another usecase for this kind of scenario is Deployd's advanced queries. They end up with JSON object that takes whole query string.

It could be defined in OAPI as plain string, but we would lose power of deep validation.
It could be defined in OAPI as schema object, which would give us the power of deep validation.

The first @handrews's comment with queryContent as new in value which forces usage of content details looks like a promising minimal proposal to cover this issue.

[RubenGarcia]

According to
https://swagger.io/specification/#parameterContent
and
https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.2.md#parameterContent
you can use a combination of style and explode to select how the url looks like.

From the table of examples, since I want "in: query", I can choose style from form,space delim, pipedelim or deep object.

If I now use an array of size 1, and spaceDelimited, and explode=false, I should be getting the value without the variable name or the equals sign.

However, https://editor.swagger.io/ still adds the "=". According to the messages above, it looks like the table in https://swagger.io/specification/#parameterContent is wrong in various places, and the parameter name is always included.

The issue is that
https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.2.md#parameterContent
by definition, defines the standard and cannot be "wrong". So it's the implementation at swagger.io that is in error.

[garymazz]

I would like to see this feature as well.. RAML handles it using regex, which also brings other valuable capabilities for early "rpc on http" apis. Swagger/OAS can adopt the same technique as RAML; using regex across field keys. Currently, Swagger/OAS applies regex is to field values, it should be a short hop for keys.

[keyz182]

This would also be useful for JSONAPI exposed through OpenAPI. ?filter[foo]=bar is the format for JSONAPI filtering, (though the specifics are not defined, and left to implementations). In this case, foo could be just the field name, or some complex thing like foo.contains or foo.prop[2] or whatever.

What I'd like to be able to do, similar to @RubenGarcia's idea, is some way to allow the user to add multiples, but instead of just specifying the right hand side, specify the left too if that makes sense?

Just illustrive - something like this:

      - in: query
        name: filter[{my_query}]
        required: false
        schema:
          type: string

That would allow adding multiple params of the form filter[...]=....

We can enumerate these server side for basic types, but we also have complex nested objects, and operations, such as ?filter[inputs.datetime.gte]=242839744 - to query for the datetime property of inputs where the time is greater than or equal to 242839744. With all of these enumerated, it can easily end up with hundreds, if not thousands of combinations, which gets completely over the top quite quickly. Cleaner (for us) to just provide a template, and document.

[hkosova]

@keyz182 your scenario can be described by using a deepObject-style parameter:

      - in: query
        name: filter
        required: false
        schema:
          type: object
          additionalProperties: true
          example:
            foo: bar
            inputs.datetime.gte: 242839744
        style: deepObject

        # The example translates to:
        # ?filter[foo]=bar&filter[inputs.datetime.gte]=242839744

[keyz182]

@hkosova Oh wow, that's perfect! Looks like I need to do some more RTFM :)

[handrews]

This issue has a use case of an almost-but-not-quite application/x-www-form-urlencoded format that includes "flag" parameters without a = involved.

• Not clear how to describe a “flag” parameter in OpenAPI #1782

[handrews]

As noted in a comment there, I think implementing this would also allow more experimentation with deepObject-like formats that handle more use cases.

This could also be explicitly called out as an extension point so that tools can provide a hook for 3rd-party support, rather than adding an open-ended mandatory tooling requirement.

[handrews]

@OAI/tsc review request: Let's approve this for 3.2, and I can write a PR to add in: queryString as an option. It would solve numerous other issues. There are two major use cases:

Parity with form-urlencoded request bodies

in: queryString
content:
  application/x-www-form-urlencoded: {...}

This provides exact parity with how application/x-www-form-urlencoded request bodies work, which would also solve at least the following:

• How to disallow unknown query parameter? #2511 (by using additionalProperties normally in the schema)
• Supporting of one separate 'required' option for query parameters #2347 (by using required normally in the schema)
• Aliasing query parameters #2298 (by using $ref normally)
• Query string parameter interdependencies (issues moved to moonwalk)

Ability to use (and experiment with) arbitrary query string formats

This is the original motivation for this issue. This would also allow us to recommend that people who want new/complex query serializations define a media type and a way to model them in schemas as an extension, particularly if we implement #3771 (Registry of media type modeling approaches). This could arguably allow us to close (by deferring to extensions) issues including:

• support odata? #230
• Support for ordered array query params #883
• Are indexes in the query parameter array representable? #1501
• How to describe urls with ordered indexed array? #1504
• Mixed type array with array of objects in location path  #2276
• Proposal: dotDelimited style for query serialization. #3084

All of these request new serialization approaches based on either existing formats that we can't easily describe (OData) or on a variety of other formats used elsewhere, or just directly requested for one reason or another. It's unlikely that we'd deal with any of these because this area is complex enough as it is, but given users full control over the query string gives them a way to solve it themselves.

[lornajane]

Discussed in this week's TDC meeting and agreed that this idea has lots of merit. We requested @handrews to work this into a formal proposal to target 3.2