[meta] Relay Core API

[josephsavona]

Relay Core API

The current (public) Relay API is primarily focused on high-level integration with React via RelayContainer and RelayRenderer. Internally at Facebook, we make use of Relay internals in a variety of ways, notably to implement several variations on server "rendering" and for GraphQL subscriptions. We've also experimented with things like running parts of Relay in a web worker.

These types of features are currently somewhat difficult to implement in open source due to the restricted set of functionality exposed in the public API. This issue is to track work toward a new, more modular public API. We plan to strike a balance between overly monolithic and overly decomposed, and split Relay into two main parts:

• Relay Core: imperative API for fetching, observing, and updating data.
• React/Relay: React integration in the form of RelayContainer and RelayRenderer, implemented purely in terms of the public Relay Core API.

Goals

Our goal with this approach is to enable more experimentation in user space, while retaining the ability to efficiently fetch data for entire view hierarchies in a single round trip.

Examples of things that should be possible with this new API:

• Server rendering (or pre-fetching data on the server and rendering on the client).
• Implementing real-time updates (subscriptions) in user-space.
• Offline persistence.
• Implementing your own alternatives to RelayContainer/RelayRenderer.
• Using Relay Core with non-React view libraries.

As mentioned, we're already doing some of these things internally: what's involved here is exposing more of the internal APIs in a slightly more user-friendly way, documenting them, and providing examples.

Strawman API

Below is a possible list of core primitives along with examples of how they could be composed:

• readFragmentData - get the data for a fragment give some node id
• observeFragmentData - like read, but get notifications as data changes
• readQueryData - get the data for a query
• observeQueryData - like read, but get notifications of changes
• buildQueryFromFragment - turn a fragment into a query (useful for refetching select parts of data)
• canResolveQueryLocally - can a query be resolved from cached data?
• fetchQueryFromCache - load data from persisted storage. returns a payload that can be applied with writeQueryPayload
• fetchQueryFromNetwork(forceFetch) - fetch data from the network, optionally riffing against the cache. returns a payload that can be applied with writeQueryPayload
• sendOperation(operation) - send a mutation/subscription to the server, asynchronously get the response
• createOperation - given a mutation/subscription description and props, construct a mutation transaction
• createQuery - construct a runtime query descriptor from a Relay.QL expression
• createFragment - construct a runtime fragment descriptor from a Relay.QL expression
• writeQueryPayload(query, payload) - write data for a payload into the store, triggering notifications of observers on data that changes
• writeOptimtimisticOperationPayload(operation, payload, configs) - write an optimistic response to the store. would return a way to undo/revert the write.
• writeOperationPayload(operation, payload, configs) - write a mutation/subscription response directly to the main store

Example: Fetching Data

• Check if the data can be fulfilled from the cache with canResolveQueryLocally.
• If not, fetch the data with fetchQueryFromNetwork and then apply the result with writeQueryPayload
• Render components, resolving their data with readFragment and setting up subscriptions with observeFragment to know when to update components

Example: Ad-hoc Subscriptions

• Listen for server events e.g. via web sockets
• Apply updates via writeOperationPayload

Example: Server-less Relay

• Create a schema description using the GraphQL schema definition language (type User {name: String}) so that you can compile Relay.QL queries
• Write data into the store with writeQueryPayload or writeOperationPayload
• Render components with readFragment and listen for updates with observeFragment.

Tasks

• Implement the primitive methods described above
• Implement existing Relay functionality (especially RelayContainer/RelayRenderer) via the core methods
• TBD

[devknoll]

Get the set of queries from a Relay.Route & Relay.Container pair (getRelayQueries)

Where Relay.Container is defined as just the structure with fragments, variables, initialVariables, and prepareVariables, and not the React HOC, right?

[josephsavona]

Yeah, something like RelayContext.prototype.getQueries(new MyRoute(...), MyContainer)

[devknoll]

If it's separated from React, is there a better name for the data structure besides Relay.Container? Without the React component, there's not really anything being contained -- it's more of a query definition I guess...

edit: How about something like QueryFragments (to go along with RelayQueryRoots)?

[josephsavona]

I see what you mean - the method to create a query set from route + container can probably live in react-relay, separate from core.

[devknoll]

Wait, it seems like it might be beneficial to have route + container* config in core 😄 I just think the concept is broader than the React implementation.

Consider the previous mixin implementation, where it wasn't a HoC, but still took some fragments and a route to generate a query set (as far as I know at least 😉) At least to me it seems like a fundamental and common operation 😄

* Probably just not called container.

[josephsavona]

I'm thinking that core should have a notion of a generic DataContainer that contains most of the logic for reading out data for fragments, observing changes to that data, updating variables, etc (all the stuff that Relay Container passes to the inner component). Then RelayContainer could be a lightweight wrapper over this.

[syrusakbary]

I think this will really help GraphQL and Relay go to the next level. Not only helping adoption in React but also other libraries like Vue.js or Angular.

Could be great if are separated repos so we can reuse and port the code easily to other platforms. Python? Go? Swift?
Looking forward for seeing how this evolves!

[andimarek]

I created a PR recently (#721) which is more or less a Implementation of @josephsavona DataContainer.

That PR is based on the current version of Relay, which means there is no clear notion of "Relay Core". In that sense it doesn't resolve this issue here.

[andimarek]

I would like to help moving this topic forward, if possible:

For example we could start moving some aspects of RelayContainer, which are React independent into own modules (e.g. creation of the FragmentPointers/QuerSet).

I'm happy to create a PR for this.
What do you think?

[josephsavona]

@andimarek thanks for your interest in this. Because this change is so fundamental to Relay, the core/react split is something that the Relay core team will be working on in the next few weeks and months (Note that we've documented in the roadmap which projects the core team is focusing on). We use Relay in a lot of complex ways (server rendering, deferred queries, disk caching, subscriptions, and prototypes of future enhancements) and ultimately we have the best context for making this change in a way that is compatible with these features and the projects that are using them.

That said, we want to support the community in developing Relay integrations. The best way to contribute is exactly what you did in your PR: build an example integration via the current API, and find & document what's missing in order to achieve the integration more cleanly.

[andimarek]

@josephsavona Thanks for this quick response. I understand your position and that you want to make sure that those fundamental changes are going into the right decision.

Because my PR #721 will not be merged soon I might end up maintaining a temporary fork of relay to be able to use it without React. Of course I'm happy to contribute back the lessons learned.

[josephsavona]

@andimarek You might also want to take a look at how https://github.com/denvned/isomorphic-relay and https://github.com/relay-tools/react-router-relay are implemented - rather than fork Relay, an alternative is to simply pull in the parts that you need from Relay.

[andimarek]

@josephsavona Thanks for these hints... I will check it out.

[andimarek]

Fyi: Based on my PR (#721) I created a modified version Relay which is not longer dependent on React and provides a generic Container:

https://github.com/andimarek/generic-relay

I think this will be just a temporary project and will be deprecated once this Issue here is resolved. It's clearly marked as experimental and I have put a clear note on the beginning of the README, which links back to here.

I hope to gain some more insights and maybe get some feedback from some early adopters about how to use Relay without React to help with this Issue here.

[josephsavona]

Note: updated the description to reflect our current thinking. Feedback welcome!

[josephsavona]

@andimarek If you have time, it would be great to get your feedback on this proposal.

[andimarek]

@josephsavona sure ... I will give feedback soon.

[andimarek]

@josephsavona Overall it looks very good.
As a general goal I would also hope that relay-core will be a new npm module, with no dependency on react.

About the proposed API: I think it covers already a lot.

I am not so sure about the cache specific methods: It could be more a implementation detail of the core and only visible through some flag (e.g.useCacheOnly) instead of specific methods like fetchQueryFromCache.
Based on my generic-relay experiment and the integration in Angular the API seems a bit more low-level than needed. (Not saying this is wrong: Better this way than a too high level API, imho).

The server-less rendering is great, because it will help to have a central Relay-Store even when some data is fetched over the network outside of Relay.

What about methods for all running queries/tasks? This could also be useful.

[syrusakbary]

Any update/working progress on this?

[wincent]

@syrusakbary: yes, check out the meeting notes in this repo. You'll find various references to "prototype", which all refer to our work on building out this new API.

No timeline yet for any of this stuff to be published, but we're working towards it and plan to provide more detail as we get closer. For now, the meeting notes are the best place to get a sense of what we're working on.

[syrusakbary]

Thanks for the quick reply @wincent!

[josephsavona]

Rolling into #1369