[RFC] Flat chain syntax

[mnpenner]

Often times we want to fetch just a single field from a nested relation. e.g.

{
  clients(limit: 5) {
    id
    programs {
      shortName
    }
  }
}

It would be nice if there was a syntax to allow getting back a flat array of program shortNames instead of an array of objects. e.g.

{
  clients(limit: 5) {
    id
    programs: programs.shortName
  }
}

This would return something like:

{
  "data": {
    "clients": [
      {
        "id": 2,
        "programs": ["ABC","DEF"]
      },
      {
        "id": 3,
        "programs": ["ABC", "XYZ"]
        ]
      }
    ]
  }
}

Giving it an alias could either be mandatory, or it could default to using the outer field name (default to "programs", "shortName" would be discarded).

A library like GraphQL-JS could handle the normalization of array-of-objects to flat-array-of-strings automatically, so there wouldn't be any additional work for someone implementing a schema.

[geospofford-anaplan]

+1 The same would be convenient for getting a single field back from a non-list object field as well.

[benwilson512]

The downside here is that it's no longer definite what type a given key in the response has. programs in your case used to have a type that was a list of objects. Now it can be a list of objects OR a list of any field on that object.

[mnpenner]

@benwilson512 But I can already alias something else to programs and 'change its type' so to speak. Where would this be a problem? And if it is a problem, we could always go with the forced-alias solution, although I kind of do like the using the outer field name as the default.

Most of the places I would want to use this would be like client names, user names, program names, etc. in which case "clients", "users" and "programs" are already good defaults. I might also want to use it for lists of IDs, in which case I'd probably alias it to client_ids instead.

[mnpenner]

Another use-case would be to flatten Relay-style result sets for when you don't care about pagination.

e.g. instead of

{
  user {
    id
    name
    friends(first: 10, after: "opaqueCursor") {
      edges {
        cursor
        node {
          id
          name
        }
      }
      pageInfo {
        hasNextPage
      }
    }
  }
}

You would be able to do

{
    user {
        id
        name
        friends(first: 10).edges.node.name
    }
}

To get a user and his first 10 friends' names as a flat array, or

{
    user {
        id
        name
        friends.edges.node {
            id,
            name
        }
    }
}

To get an array of friend objects.

[rmosolgo]

-1 for adding syntax, something I love about GraphQL is that there isn't much to the syntax

[r4j4h]

I know referring to another library is kind of crappy, and this doesn't help with your symbolic extension, but would d3.nest help you reformat the QL responses into a more digestible format? Here's a handy demonstration of its re-layout-ing capabilities.

[mnpenner]

@r4j4h Those examples are weird. It's creating objects with two properties, key and values. If anything, I'd want the key to be the actual key for the inner object. I'm guessing that's the format needed for the D3 charts? Doesn't seem too useful for general-purpose.

Reformatting on the client isn't terrible difficult, I was just hoping to remove some of the boilerplate from the client and make it a little easier for novices. One of the nice things about GraphQL is that it's easy to read and write, which means you can give it to non-techies and they can pull out some basic data in a without having to hassle you. Having excessive nesting in the output makes it harder to read.

[He-Pin]

I have to say ,this would be nice,if this is a bad thing ,why js have it?
@mnpenner Even this is not implemented or defined,I think we will still implement it for the real case usage.

[He-Pin]

I love this one,especially I think we could scale it to larger scope.
This would help when we only have one field which is deeply en-scoped.eg
myD:a{b{c{d}}} could be rewrite to myD:a.b.c.d I would call this a Selection.

I know this would complicate the current parser implement,but it's very convenient,like the lens.

[leebyron]

This is a really interesting proposal, thanks for writing it up.

I'm curious if you're finding cases where you couldn't express something correctly before, or if this is just purely a matter of convenience? So far GraphQL has avoided convenience features as a means of avoiding feature creep, but it's also not a strict rule. Put another way: adding syntax is "expensive" and the value of having it should be well worth the cost of complexity.

Another potential concern is how this should behave when following various types. I'm not sure @benwilson512's concern is exactly correct, but it's directionally correct. Because everything in GraphQL is typed, we're not giving up type information by allowing this kind of chaining, but we could be making it more complicated. For example:

{
  clients(limit: 5) {
    id
    programsShortName: programs.shortName
  }
}

If we know programs is going to return [Program] where Program has a field called shortName of type String then we can certainly know that programs.shortName would return [String].

I'm curious if there are cases where Nullable / Non-nullable and List / Singlar typed fields can be written in this form to come up with an ambiguous or problematic type?

[mnpenner]

I'm curious if you're finding cases where you couldn't express something correctly before, or if this is just purely a matter of convenience?

Convenience. I want to keep my API flexible without having to implement a dozen variants of everything.

I could implement programShortNames, programIds and programs as fields of clients (which are all pretty commonly used) or if GraphQL allowed it, I'd just have to implement the latter and I'd get programs.shortName and programs.id for free. Well, either that or force the client to do the reformatting on their end.

I figured it would be possible to resolve the type as you suggested. I can't think of a scenario where this would be ambiguous.

Let's run through the scenarios:

programs	shortName	programs.shortName
[Program]	String	[String]
[Program!]	String	[String]
[Program!]!	String	[String]!
[Program]!	String	[String]!
[Program]	String!	[String]
[Program!]	String!	[String!]
[Program!]!	String!	[String!]!
[Program]!	String!	[String]!

The only funny scenarios are when the program can be null but the string cannot be null -- in such cases the string would be forced to be null if the program was null.

[sorenbs]

One additional aspect is performance. We expose two versions of our api - one that is relay compatible and one that is human readable. For one particular query the relay api returned 30 mb of data and the simple api returned 400 kb simply by avoiding the connection boilerplate.

This is the difference between the browser freezing for tens of seconds vs barely being noticeable. This proposal would allow us to get the same level of client side performance from the relay api.

[stubailo]

I think this is a great point - being able to short-circuit paths would make deeply nested schemas much more palatable.

[calebmer]

One additional aspect is performance. We expose two versions of our api - one that is relay compatible and one that is human readable. For one particular query the relay api returned 30 mb of data and the simple api returned 400 kb simply by avoiding the connection boilerplate.

If we had this in the spec would it really help the Relay case? Unless Relay changes so that it doesn’t need a cursor for every node, this syntax won’t be used. Relay could also change the spec to add a nodes field to connections that returns an array of the node object without cursors as is done in PostGraphQL and the SWAPI example API.

Also, are the numbers regarding connection boilerplate gzipped? I’m guessing connection boilerplate is a smaller problem (size wise) then the repeated data that is a product of denormalization. However the repeated data problem is mitigated by gzipping.

[sorenbs]

Even if relay doesn't add support, it would still make it possible to have just one relay compliant api and use relay for most things, but fall back to plain http request in edge cases like this. In our case this request was made from a separate admin tool that didn't use relay anyway.

I don't have the numbers, but in our case the primary issue seemed to be parsing the data (browser was stalled several seconds) and not network transfer as gzip indeed mostly solves that.

[grahamegrieve]

I think this is very important. If you are using graphQL for a simple restful API, navigating the nesting is not such an onerous task. But if you're getting data and processing it (an R client, for instance), then working over oddly shaped data is onerous... where as writing a graphQL query is easy and re-usable. and graphQL includes nearly everything you want... except this. (well, everything I have discovered that is wanted, and I haven't imagined anything else - yet?). And it would be very onerous indeed for the server to try and predict all the flattening in advance...

for now, I have done what #268 proposes in my implementation - and that's probably enough for me. My server will cause an error if types get mixed. This would be a very useful feature to formally document if graphQL is going to be used inside R

[daemonsy]

One very weird use case I have is pushing for the adoption of GraphQL and this might help lower the barrier. It's much higher cost and surface area to get a server running and changing the API consumers at the same time to issue queries.

The approach I've taken is to replace endpoints of a Rails app with GraphQL. For example, if I had a Resource and each resource had an Owner (sorry I have to obfuscate it), I'd like to type them and express this relationship

{ 
  resource {
    name
    owner {
      name
    }
  }
}

However, a lot of our legacy code are flattening these relationships, ahem, out convenience.

// current controller response for a Resource
{
  id: 13,
  name: "Needle",
  owner_first_name: "Arya",
  owner_last_name: "Stark"
}

The availability of these syntax will help me write some GQL queries that fit the shape of these responses without having to add legacy fields to the object type AND avoid having to change the frontend right away.

No burning desire to have this, just another point for consideration.

[loganpowell]

Another case for allowing a flattening syntax would be for standard formats such as GeoJSON (e.g., FeatureCollection), where - in order to be consumed by standard libraries - would need to have the format in a specific shape. Here's a good article from a SME (formerly mapbox):

The properties attached to a feature can be any kind of JSON object. That said, given the fact that no other prominent geospatial standard supports nested values, usually the properties object consists of single-depth key⇢ value mappings.

This is an immediate need of mine as I am currently attempting to wrap a number of US Census api endpoints to provide convenience to developers who wish to create maps using our data.

[loganpowell]

Also, found a library for those who need this now:

https://www.fourkitchens.com/blog/development/graphql-leveler-controlling-shape-query/

connecting to another related issue: #174

[leebyron]

Marking this as Needs Champion. I still think this is an interesting RFC, but it needs work to progress.

[NuckChorris]

What would be expected of a champion for this? I think this would remove my only complaint about GraphQL so If I can find the time to champion it I'm definitely down

[mike-marcacci]

While I’ve never used it, the graphql-lodash library achieves this and much more through directives. That may be a good option for those looking to experiment with this functionality to find its value and limitations.

[jimisaacs]

It's been a while since anyone has posted here, and I was wondering what the sentiment is today versus 3 years ago? Personally I think this was and still is a great idea. I'm interested in championing it, but also curious what that would entail.

[benjie]

Hi @jimisaacs; to get a feature like this into the GraphQL Spec requires advancing it through the 4 RFC stages which normally will take at least 4 working groups. This one is already at RFC0 though, so that's one step complete! You can read more about the stages here: https://github.com/graphql/graphql-spec/blob/main/CONTRIBUTING.md#rfc-contribution-stages Essentially you start with a proposal (like this), gather feedback and use cases, then write this up into concrete spec edits and it's often beneficial (but not mandatory) to maintain an RFC document in this folder with details of what's being proposed, why, and what decisions have been made: https://github.com/graphql/graphql-wg/tree/main/rfcs. Later you'll also push forward an implementation of it in graphql-js and hopefully convince another implementation to adopt it too; this gives implementors a chance to find any issues. Then we discuss it some more and if we get consensus, in it goes!

Before embarking on this quest, I recommend you read the guiding principles: https://github.com/graphql/graphql-spec/blob/main/CONTRIBUTING.md#guiding-principles because you'll have to come up with convincing reasons why this change should be made; in particular I think "Simplicity and consistency over expressiveness and terseness" and "Favor no change" will be the biggest challenges for this proposal.

Should you wish to just gather current opinions, I recommend that you add yourself and this as a short (10-15m) agenda item to the next WG you can attend, e.g. https://github.com/graphql/graphql-wg/blob/main/agendas/2022/2022-06-02.md. You need to agree to the CLA and another agreement or two. You may also choose to make some spec edits via a PR and propose elevating this to RFC1 at the WG.

If you need any more guidance, you can reach out to me either here or via the GraphQL Discord https://discord.graphql.org/ in the #wg channel.

[ADTC]

I think since the output of flat-chained syntax would violate GraphQL principles, (edit: actually it doesn't) it would make more sense for us to build libraries or functions (outside core GraphQL) to transform flat-chained syntax to regular syntax, then transform the regular output to the expected output of flat-chained syntax.

Any such thing?

Edit: Found these three in the comments above:

• http://bl.ocks.org/phoebebright/raw/3176159/ (https://github.com/d3/d3-array/blob/v3.2.0/README.md#transformations)
• https://www.fourkitchens.com/blog/development/graphql-leveler-controlling-shape-query/ (https://github.com/chasingmaxwell/graphql-leveler)
• https://github.com/IvanGoncharov/graphql-lodash

[drkibitz]

violate GraphQL principles

How?

[ADTC]

violate GraphQL principles

How?

I might have misunderstood while reading this before. Now I realize what he meant is that the custom directive violates the principle by changing the shape of response.

Flat chain syntax itself doesn't violate that.

[Xample]

one major drawback I see here is that flattening the structure might also omit some ids which are mandatory to cache (and update) the result properly on the client side client side
. For this reason, my understanding is that it is safer to use another lib in a latter stage to pick the key we need (such as lodash's map and get). If used properly, if the child property gets updated within a request, the virtual property will too. Of course, we could imagine having some magic path picker which would take all the "ID" property along the path, but this would be "too magic" to be an acceptable design.

[ThaDaVos]

one major drawback I see here is that flattening the structure might also omit some ids which are mandatory to cache (and update) the result properly on the client side client side . For this reason, my understanding is that it is safer to use another lib in a latter stage to pick the key we need (such as lodash's map and get). If used properly, if the child property gets updated within a request, the virtual property will too. Of course, we could imagine having some magic path picker which would take all the "ID" property along the path, but this would be "too magic" to be an acceptable design.

I fully get your statement about caching - but isn't this something the implementing side (the client side which chooses to use the flat chaining syntax) to keep it mind when using the syntax?

I still would love "flat chain syntax" to be part of the spec as it's a great feature to have - GraphQL's main point, as far as I remember, is asking and retrieving what you want and need - getting the whole 50 levels connection structure only because you cannot flat it, may be considered as unneeded and using "flat chain syntax" to get the single field from all the way down there is what's needed.