Introduce Data providers (#2644)

Signed-off-by: Jan Potoms <2109932+Janpot@users.noreply.github.com>
Co-authored-by: Marija Najdova <mnajdova@gmail.com>
Co-authored-by: Pedro Ferreira <10789765+apedroferreira@users.noreply.github.com>

.eslintrc.js

@@ -34,6 +34,10 @@ module.exports = {
  message:
  'Avoid kitchensink libraries like lodash-es. We prefer a slightly more verbose, but more universally understood javascript style',
  },
+ {
+ name: 'react-query',
+ message: 'deprecated package, use @tanstack/react-query instead.',
+ },
  ],
  patterns: [
  {

docs/data/pages.ts

@@ -44,6 +44,10 @@ const pages: MuiPage[] = [
  pathname: '/toolpad/concepts/custom-functions',
  title: 'Custom functions',
  },
+ {
+ pathname: '/toolpad/concepts/data-providers',
+ title: 'Data providers',
+ },
  ],
  },
  {
@@ -163,14 +167,18 @@ const pages: MuiPage[] = [
  pathname: '/toolpad/reference/api/functions-group',
  subheader: 'Functions',
  children: [
- {
- title: 'createFunction',
- pathname: '/toolpad/reference/api/create-function',
- },
  {
  title: 'createComponent',
  pathname: '/toolpad/reference/api/create-component',
  },
+ {
+ title: 'createDataProvider',
+ pathname: '/toolpad/reference/api/create-data-provider',
+ },
+ {
+ title: 'createFunction',
+ pathname: '/toolpad/reference/api/create-function',
+ },
  {
  title: 'getContext',
  pathname: '/toolpad/reference/api/get-context',

docs/data/toolpad/concepts/data-providers.md

@@ -0,0 +1,119 @@
+# Data Providers
+
+<p class="description">Bring tabular data to the frontend with server-side pagination and filtering.</p>
+
+Toolpad functions are great to bring some backend state to the page, but they fall short when it comes to offering pagination and filtering capabilities from the server. Toolpad offers a special construct to enable this use case: Data providers. Data providers abstract server-side collections. They could be database tables, REST APIs, or any data that represents a set of records that share a common interface. Data providers are defined as server-side objects and can be directly connected to a data grid to make it fully interactive.
+
+Follow these steps to create a new data provider:
+
+1. Drag a data grid into the canvas
+
+2. Under its **Row Source** property, select the option **Data Provider**.
+
+{{"component": "modules/components/DocsImage.tsx", "src": "/static/toolpad/docs/concepts/data-providers/rows-source.png", "alt": "Select data provider row source", "caption": "Select data provider row source", "zoom": false, "width": 297}}
+
+3. Click the data provider selector and choose **Create new data provider**.
+
+{{"component": "modules/components/DocsImage.tsx", "src": "/static/toolpad/docs/concepts/data-providers/create-data-provider.png", "alt": "Create data provider", "caption": "Create data provider", "zoom": false, "width": 294}}
+
+4. Name the new data provider and click **Create**
+
+{{"component": "modules/components/DocsImage.tsx", "src": "/static/toolpad/docs/concepts/data-providers/create-data-provider-dialog.png", "alt": "Create data provider dialog", "caption": "Create data provider dialog", "zoom": false, "width": 490}}
+
+5. Use the code button to open your code editor with the data provider backend.
+
+{{"component": "modules/components/DocsImage.tsx", "src": "/static/toolpad/docs/concepts/data-providers/open-editor.png", "alt": "Open data provider editor", "caption": "Open data provider editor", "zoom": false, "width": 272}}
+
+6. A data provider that iterates over a static list could look as follows:
+
+ ```tsx
+ import { createDataProvider } from '@mui/toolpad-core/server';
+ import DATA from './movies.json';
+
+ export default createDataProvider({
+ async getRecords({ paginationModel: { start = 0, pageSize } }) {
+ const records = DATA.slice(start, start + pageSize);
+ return { records, totalCount: DATA.length };
+ },
+ });
+ ```
+
+## Pagination
+
+The data provider supports two styles of pagination. Index based, and cursor based pagination.
+
+### Index based
+
+This is the strategy your data is paginated by when it returns data based on a page number and page size. The `getRecords` method will receive `page` and `pageSize` values in it `paginationModel` parameter and returns a set of records representing the page. Index based pagination is the default but you can explicitly enable this by setting `paginationMode` to `'index'`.
+
+```tsx
+export default createDataProvider({
+ paginationMode: 'index',
+ async getRecords({ paginationModel: { start = 0, pageSize } }) {
+ const { page, totalCount } = await db.getRecords(start, pageSize);
+ return {
+ records: page,
+ totalCount,
+ };
+ },
+});
+```
+
+### Cursor based
+
+This is the strategy your data is paginated by when it returns data based on a cursor and a page size. The `getRecords` method will receive `cursor` and `pageSize` values in its `paginationModel` parameter and returns a set of records representing the page. You indicate the cursor of the next page with a `cursor` property in the result. Pass `null` to signal the end of the collection. You can enable Cursor based pagination by setting `paginationMode` to `'cursor'`.
+
+The `cursor` property of the `paginationModel` is `null` when Toolpad fetches the initial page. Any result set returned from the `getRecords` function must be accompanied with a `cursor` property, a string which contains a reference to the next page. This value will be passed as the `cursor` parameter in the `paginationModel` when fetching the subsequent page. Return `null` for this value to indicate the end of the sequence.
+
+```tsx
+export default createDataProvider({
+ paginationMode: 'cursor',
+ async getRecords({ paginationModel: { cursor = null, pageSize } }) {
+ const { page, nextPageCursor, totalCount } = await db.getRecords(
+ cursor,
+ pageSize,
+ );
+ return {
+ records: page,
+ cursor: nextPageCursor,
+ totalCount,
+ };
+ },
+});
+```
+
+## Filtering 🚧
+
+:::warning
+This feature isn't implemented yet. It's coming.
+:::
+
+## Sorting 🚧
+
+:::warning
+This feature isn't implemented yet. It's coming.
+:::
+
+## Row editing 🚧
+
+:::warning
+This feature isn't implemented yet. It's coming.
+:::
+
+## Row creation 🚧
+
+:::warning
+This feature isn't implemented yet. It's coming.
+:::
+
+## Deleting rows 🚧
+
+:::warning
+This feature isn't implemented yet. It's coming.
+:::
+
+## API
+
+See the documentation below for a complete reference to all of the functions and interfaces mentioned in this page.
+
+- [`createDataProvider`](/toolpad/reference/api/create-data-provider/)

docs/data/toolpad/reference/api/create-data-provider.md

@@ -0,0 +1,91 @@
+# createDataProvider API
+
+<p class="description">Define a backend to load server-side collections.</p>
+
+## Import
+
+```jsx
+import { createDataProvider } from '@mui/toolpad/server';
+```
+
+## Description
+
+```jsx
+import { createDataProvider } from '@mui/toolpad-core/server';
+import DATA from './movies.json';
+
+export default createDataProvider({
+ async getRecords({ paginationModel: { start = 0, pageSize } }) {
+ const records = DATA.slice(start, start + pageSize);
+ return { records, totalCount: DATA.length };
+ },
+});
+```
+
+Data providers expose collections to the Toolpad frontend. They are server-side data structures that abstract the loading and manipulation of a backend collection of records of similar shape. They can be directly connected to data grids to display the underlying data.
+
+## Parameters
+
+- `config`: [`DataProviderConfig`](#dataproviderconfig) An object describing the data provider capabilities
+
+## Returns
+
+An object that is recognized by Toolpad as a data provider and which is made available to the front-end.
+
+## Types
+
+### DataProviderConfig
+
+Describes the capabilities of the data provider.
+
+**Properties**
+
+| Name | Type | Description |
+| :---------------- | :----------------------------------------------------- | :------------------------------------------------------ |
+| `paginationMode?` | `'index' \| 'cursor'` | Declares the pagination strategy of this data provider. |
+| `getRecords` | `async (params: GetRecordsParams) => GetRecordsResult` | Responsible for fetching slices of underlying data. |
+
+### GetRecordsParams
+
+**Properties**
+
+| Name | Type | Description |
+| :----------------- | :---------------- | :------------------------------------------------------- |
+| `paginationModel?` | `PaginationModel` | The pagination model that describes the requested slice. |
+
+### PaginationModel
+
+- `IndexPaginationModel` when `paginationMode` is set to `'index'`.
+- `CursorPaginationModel` when `paginationMode` is set to `'cursor'`.
+
+### IndexPaginationModel
+
+**Properties**
+
+| Name | Type | Description |
+| :--------- | :------- | :------------------------------------------------------ |
+| `start` | `number` | The start index of the requested slice requested slice. |
+| `pageSize` | `number` | The length of the requested slice. |
+
+### CursorPaginationModel
+
+**Properties**
+
+| Name | Type | Description |
+| :--------- | :------- | :---------------------------------------------------------------------- |
+| `cursor` | `number` | The cursor addressing the requested slice. `null` for the initial page. |
+| `pageSize` | `number` | The length of the requested slice. |
+
+### GetRecordsResult
+
+| Name | Type | Description |
+| :------------ | :--------------- | :-------------------------------------------------------------------------------------------------------------------------------------------- |
+| `records` | `any[]` | The start index of the requested slice requested slice. |
+| `totalCount?` | `number` | The length of the requested slice. |
+| `cursor?` | `string \| null` | Used when `paginationMode` is set to `cursor`. It addresses the next page in the collection. Pass `null` to signal the end of the collection. |
+
+## Usage
+
+:::info
+See [data providers](/toolpad/concepts/data-providers/)
+:::

docs/data/toolpad/reference/api/index.md

@@ -5,5 +5,6 @@
 ## Functions
 
 - [createComponent](/toolpad/reference/api/create-component/)
+- [createDataProvider](/toolpad/reference/api/create-data-provider/)
 - [createFunction](/toolpad/reference/api/create-function/)
 - [getContext](/toolpad/reference/api/get-context/)