Add support for embedding and prefetching

docs/DataProviderWriting.md

@@ -466,19 +466,143 @@ A simple react-admin app with one `<Resource>` using [guessers](./Features.md#gu
 
 ## The `meta` Parameter
 
-All data provider methods accept a `meta` parameter. React-admin core components never set this `meta` when calling the data provider. It's designed to let you pass additional parameters to your data provider.
+All data provider methods accept a `meta` query parameter and can return a `meta` response key. React-admin core components never set the query `meta`. It's designed to let you pass additional parameters to your data provider.
 
-For instance, you could pass an option to embed related records in the response:
+For instance, you could pass an option to embed related records in the response (see [Embedded data](#embedded-data) below):
 
 ```jsx
-const { data, isPending, error } = useGetOne(
+const { data } = await dataProvider.getOne(
     'books',
-    { id, meta: { _embed: 'authors' } },
+    { id, meta: { embed: ['authors'] } },
 );
 ```
 
 It's up to you to use this `meta` parameter in your data provider.
 
+## Embedded Data
+
+Some API backends with knowledge of the relationships between resources can [embed related records](./DataProviders.md#embedding-relationships) in the response. If you want your data provider to support this feature, use the `meta.embed` query parameter to specify the relationships that you want to embed.
+
+```jsx
+const { data } = await dataProvider.getOne(
+    'posts',
+    { id: 123, meta: { embed: ['author'] } }
+);
+// {
+//    id: 123,
+//    title: "Hello, world",
+//    author_id: 456,
+//    author: { id: 456, name: "John Doe" },
+// }
+```
+
+For example, the [JSON server](https://github.com/typicode/json-server?tab=readme-ov-file#embed) backend supports embedded data using the `_embed` query parameter:
+
+```txt
+GET /posts/123?_embed=author
+```
+
+The [JSON Server Data Provider](https://github.com/marmelab/react-admin/tree/master/packages/ra-data-json-server) therefore passes the `meta.embed` query parameter to the API:
+
+```tsx
+const apiUrl = 'https://my.api.com/';
+const httpClient = fetchUtils.fetchJson;
+
+const dataProvider = {
+    getOne: async (resource, params) => {
+        let query = `${apiUrl}/${resource}/${params.id}`;
+        if (params.meta?.embed) {
+            query += `?_embed=${params.meta.embed.join(',')}`;
+        }
+        const { json: data } = await httpClient(query);
+        return { data };
+    },
+    // ...
+}
+```
+
+As embedding is an optional feature, react-admin doesn't use it by default. It's up to you to implement it in your data provider to reduce the number of requests to the API.
+
+## Prefetching
+
+Similar to embedding, [prefetching](./DataProviders.md#prefetching-relationships) is an optional data provider feature that saves additional requests by returning related records in the response.
+
+Use the `meta.prefetch` query parameter to specify the relationships that you want to prefetch.
+
+```jsx
+const { data } = await dataProvider.getOne(
+    'posts',
+    { id: 123, meta: { prefetch: ['author'] } }
+);
+// {
+//     data: {
+//         id: 123,
+//         title: "Hello, world",
+//         author_id: 456,
+//     },
+//     meta: {
+//         prefetched: {
+//             authors: [{ "id": 456, "name": "John Doe" }]
+//         }
+//     }
+// }
+```
+
+By convention, the `meta.prefetched` response key must be an object where each key is the name of the embedded resource, and each value is an array of records.
+
+It's the Data Provider's job to build the `meta.prefetched` object based on the API response.
+
+For example, the [JSON server](https://github.com/typicode/json-server?tab=readme-ov-file#embed) backend supports embedded data using the `_embed` query parameter:
+
+```txt
+GET /posts/123?_embed=author
+```
+
+```json
+{
+    "id": 123,
+    "title": "Hello, world",
+    "author_id": 456,
+    "author": {
+        "id": 456,
+        "name": "John Doe"
+    }
+}
+```
+
+To add support for prefetching, the [JSON Server Data Provider](https://github.com/marmelab/react-admin/tree/master/packages/ra-data-json-server) extracts the embedded data from the response, and puts them in the `meta.prefetched` property:
+
+```jsx
+const dataProvider = {
+    getOne: async (resource, params) => {
+        let query = `${apiUrl}/${resource}/${params.id}`;
+        if (params.meta?.prefetch) {
+            query += `?_embed=${params.meta.prefetch.join(',')}`;
+        }
+        const { json: data } = await httpClient(query);
+        const prefetched = {};
+        if (params.meta?.prefetch) {
+            params.meta.prefetch.forEach(name => {
+                if (data[name]) {
+                    const prefetchKey = name.endsWith('s') ? name : `${name}s`;
+                    if (!prefetched[prefetchKey]) {
+                        prefetched[prefetchKey] = [];
+                    }
+                    if (!prefetched[prefetchKey].find(r => r.id === data[name].id)) {
+                        prefetched[prefetchKey].push(data[name]);
+                    }
+                    delete data[name];
+                }
+            });
+        }
+        return { data };
+    },
+    // ...
+}
+```
+
+Use the same logic to implement prefetching in your data provider.
+
 ## The `signal` Parameter
 
 All data provider queries can be called with an extra `signal` parameter. This parameter will receive an [AbortSignal](https://developer.mozilla.org/en-US/docs/Web/API/AbortSignal) that can be used to abort the request.
@@ -548,7 +672,15 @@ const { data } = dataProvider.getOne('posts', { id: 123 })
 // }
 ```
 
-This will cause the Edit view to blink on load. If you have this problem, modify your Data Provider to return the same shape for all methods. 
+This will cause the Edit view to blink on load. If you have this problem, modify your Data Provider to return the same shape for all methods.
+
+**Note**: If the `getList` and `getOne` methods use different `meta` parameters, they won't share the cache. You can use this as an escape hatch to avoid flickering in the Edit view.
+
+```jsx
+const { data } = dataProvider.getOne('posts', { id: 123, meta: { page: 'getOne' } })
+```
+
+This also explains why using [Embedding relationships](./DataProviders.md#embedding-relationships) may make the navigation slower, as the `getList` and `getOne` methods will return different shapes.
 
 ## `fetchJson`: Built-In HTTP Client
 

docs/DataProviders.md

@@ -270,6 +270,107 @@ Access-Control-Expose-Headers: X-Custom-Header
 
 This must be done on the server side.
 
+## Embedding Relationships
+
+Some API backends with knowledge of the relationships between resources can embed related records in the response.
+
+For instance, JSON Server can return a post and its author in a single response:
+
+```txt
+GET /posts/123?embed=author
+```
+
+```json
+{
+    "id": 123,
+    "title": "Hello, world",
+    "author_id": 456,
+    "author": {
+        "id": 456,
+        "name": "John Doe"
+    }
+}
+```
+
+Data providers implementing this feature often use the `meta` key in the query parameters to pass the embed parameter to the API.
+
+```jsx
+const { data } = useGetOne('posts', { id: 123, meta: { embed: ['author'] } });
+```
+
+Leveraging embeds can reduce the number of requests made by react-admin to the API, and thus improve the app's performance.
+
+For example, this allows you to display data from a related resource without making an additional request (and without using a `<ReferenceField>`).
+
+{% raw %}
+```diff
+const PostList = () => (
+-   <List>
++   <List queryOptions={{ meta: { embed: ["author"] } }}>
+        <Datagrid>
+            <TextField source="title" />
+-           <ReferenceField source="author_id" reference="authors>
+-               <TextField source="name" />
+-           </ReferenceField>
++           <TextField source="author.name" />
+        </Datagrid>
+    </List>
+);
+```
+{% endraw %}
+
+Refer to your data provider's documentation to verify if it supports this feature. If you're writing your own data provider, check the [Writing a Data Provider](./DataProviderWriting.md#embedded-data) documentation for more details.
+
+**Note**: Embeds are a double-edged sword. They can make the response larger and break the sharing of data between pages. Measure the performance of your app before and after using embeds to ensure they are beneficial.
+
+## Prefetching Relationships
+
+Some API backends can return related records in the same response as the main record. For instance, an API may return a post and its author in a single response:
+
+
+```jsx
+const { data, meta } = useGetOne('posts', { id: 123, meta: { prefetch: ['author']} });
+```
+
+```json
+{
+    "data": {
+        "id": 123,
+        "title": "Hello, world",
+        "author_id": 456,
+    },
+    "meta": {
+        "prefetched": {
+            "authors": [{ "id": 456, "name": "John Doe" }]
+        }
+    }
+}
+```
+
+This is called *prefetching* or *preloading*.
+
+React-admin can use this feature to populate its cache with related records, and avoid subsequent requests to the API. The prefetched records must be returned in the `meta.prefetched` key of the data provider response.
+
+For example, you can use prefetching to display the author's name in a post list without making an additional request:
+
+{% raw %}
+```jsx
+const PostList = () => (
+    <List queryOptions={{ meta: { prefetch: ['author'] }}}>
+        <Datagrid>
+            <TextField source="title" />
+            {/** renders without an additional request */}
+            <ReferenceField source="author_id" />
+        </Datagrid>
+    </List>
+);
+```
+{% endraw %}
+
+The way to *ask* for embedded resources isn't normalized and depends on the API. The above example uses the `meta.prefetch` query parameter. Some APIs may use [the `embed` query parameter](#embedding-relationships) to indicate prefetching.
+
+ Refer to your data provider's documentation to verify if it supports prefetching. If you're writing your own data provider, check the [Writing a Data Provider](./DataProviderWriting.md#embedded-data) documentation for more details.
+
 ## Adding Lifecycle Callbacks
 
 <iframe src="https://www.youtube-nocookie.com/embed/o8U-wjfUwGk" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen style="aspect-ratio: 16 / 9;width:100%;margin-bottom:1em;"></iframe>

docs/Features.md

@@ -851,6 +851,7 @@ React-admin takes advantage of the Single-Page-Application architecture, impleme
 - **Query Deduplication**: React-admin identifies instances where multiple components on a page call the same data provider query for identical data. In such cases, it ensures only a single call to the data provider is made.
 - **Query Aggregation**: React-admin intercepts all calls to `dataProvider.getOne()` for related data when a `<ReferenceField>` is used in a list. It aggregates and deduplicates the requested ids, and issues a single `dataProvider.getMany()` request. This technique effectively addresses the n+1 query problem, reduces server queries, and accelerates list view rendering.
 - **Opt-In Query Cache**: React-admin provides an option to prevent refetching an API endpoint for a specified duration, which can be used when you're confident that the API response will remain consistent over time.
+- **Embedded Data** and **Prefetching**: Data providers can return data from related resources in the same response as the requested resource. React-admin uses this feature to avoid additional network requests and to display related data immediately.
 
 ## Undo
 

docs/Fields.md

@@ -325,6 +325,12 @@ Then you can render the author name like this:
 <TextField source="author.name" />
 ```
 
+This is particularly handy if your data provider supports [Relationship Embedding](./DataProviders.md#embedding-relationships).
+
+```jsx
+const { data } = useGetOne('posts', { id: 123, meta: { embed: ['author'] } });
+```
+
 ## Setting A Field Label
 
 <iframe src="https://www.youtube-nocookie.com/embed/fWc7c0URQMQ" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen style="aspect-ratio: 16 / 9;width:100%;margin-bottom:1em;"></iframe>

docs/ReferenceField.md

@@ -237,6 +237,30 @@ React-admin accumulates and deduplicates the ids of the referenced records to ma
 
 Then react-admin renders the `<PostList>` with a loader for the `<ReferenceField>`, fetches the API for the related users in one call (`dataProvider.getMany('users', { ids: [789,735] }`), and re-renders the list once the data arrives. This accelerates the rendering and minimizes network load.
 
+## Prefetching
+
+When you know that a page will contain a `<ReferenceField>`, you can configure the main page query to prefetch the referenced records to avoid a flicker when the data arrives. To do so, pass a `meta.prefetch` parameter to the page query.
+
+For example, the following code prefetches the authors referenced by the posts:
+
+{% raw %}
+```jsx
+const PostList = () => (
+    <List queryOptions={{ meta: { prefetch: ['author'] } }}>
+        <Datagrid>
+            <TextField source="title" />
+            {/** renders without an additional request */}
+            <ReferenceField source="author_id" />
+        </Datagrid>
+    </List>
+);
+```
+{% endraw %}
+
+**Note**: For prefetching to function correctly, your data provider must support [Prefetching Relationships](./DataProviders.md#prefetching-relationships). Refer to your data provider's documentation to verify if this feature is supported.
+
+**Note**: Prefetching is a frontend performance feature, designed to avoid flickers and repaints. It doesn't always prevent `<ReferenceField>` to fetch the data. For instance, when coming to a show view from a list view, the main record is already in the cache, so the page renders immediately, and both the page controller and the `<ReferenceField>` controller fetch the data in parallel. The prefetched data from the page controller arrives after the first render of the `<ReferenceField>`, so the data provider fetches the related data anyway. But from a user perspective, the page displays immediately, including the `<ReferenceField>`. If you want to avoid the `<ReferenceField>` to fetch the data, you can use the React Query Client's `staleTime` option.
+
 ## Rendering More Than One Field
 
 You often need to render more than one field of the reference table (e.g. if the `users` table has a `first_name` and a `last_name` field).

examples/simple/src/comments/CommentList.tsx

@@ -148,7 +148,11 @@ const CommentMobileList = () => (
 );
 
 const CommentList = () => (
-    <ListBase perPage={6} exporter={exporter}>
+    <ListBase
+        perPage={6}
+        exporter={exporter}
+        queryOptions={{ meta: { prefetch: ['post'] } }}
+    >
         <ListView />
     </ListBase>
 );

examples/simple/src/comments/CommentShow.tsx

@@ -8,7 +8,7 @@ import {
 } from 'react-admin'; // eslint-disable-line import/no-unresolved
 
 const CommentShow = () => (
-    <Show>
+    <Show queryOptions={{ meta: { prefetch: ['post'] } }}>
         <SimpleShowLayout>
             <TextField source="id" />
             <ReferenceField source="post_id" reference="posts">

examples/simple/src/dataProvider.tsx

@@ -32,14 +32,15 @@ const addTagsSearchSupport = (dataProvider: DataProvider) => ({
             // partial pagination
             return dataProvider
                 .getList(resource, params)
-                .then(({ data, total }) => ({
+                .then(({ data, total, meta }) => ({
                     data,
                     pageInfo: {
                         hasNextPage:
                             params.pagination.perPage * params.pagination.page <
                             (total || 0),
                         hasPreviousPage: params.pagination.page > 1,
                     },
+                    meta,
                 }));
         }
         if (resource === 'tags') {