Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DataGridPro] Implement Lazy loading #5214

Merged
merged 91 commits into from
Sep 1, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
91 commits
Select commit Hold shift + click to select a range
ad7e2da
add skeleton rows/cells
DanailH Jun 6, 2022
c214c93
Merge branch 'master' of github.com:mui/mui-x into feature/DataGridPr…
DanailH Jun 10, 2022
93af334
draft version ready
DanailH Jun 15, 2022
55948da
Merge branch 'master' of github.com:mui/mui-x into feature/DataGridPr…
DanailH Jun 15, 2022
bd6f534
revert docs update
DanailH Jun 15, 2022
8db28de
resolve conflcits
DanailH Jul 6, 2022
29eafb9
PR comments
DanailH Jul 6, 2022
cc4019f
fix statics
DanailH Jul 6, 2022
0fc1bfc
polish solution
DanailH Jul 6, 2022
2fe5b7a
fix tests
DanailH Jul 6, 2022
7da81c2
enable all tests
DanailH Jul 6, 2022
82796fa
enable assertions
DanailH Jul 7, 2022
dd901a2
fix tree data test
DanailH Jul 7, 2022
a5dfc0a
add tests and make the feature experimental
DanailH Jul 13, 2022
692f4f4
fix formatting and enable tests
DanailH Jul 13, 2022
9abed60
fix test
DanailH Jul 13, 2022
0be892f
Update docs/data/data-grid/rows/rows.md
DanailH Jul 13, 2022
f565260
Update docs/data/data-grid/rows/rows.md
DanailH Jul 13, 2022
4e66d70
test
DanailH Jul 13, 2022
0fa4d64
Merge branch 'feature/DataGridPro-lazy-loading' of github.com:DanailH…
DanailH Jul 13, 2022
67b2141
Update packages/grid/x-data-grid-pro/src/hooks/features/lazyLoader/us…
DanailH Jul 13, 2022
cdacca9
Update packages/grid/x-data-grid-pro/src/hooks/features/lazyLoader/us…
DanailH Jul 13, 2022
ac2b4d8
Update packages/grid/x-data-grid-pro/src/hooks/features/lazyLoader/us…
DanailH Jul 13, 2022
27d44f8
fixes
DanailH Jul 13, 2022
e4c3a68
Merge branch 'feature/DataGridPro-lazy-loading' of github.com:DanailH…
DanailH Jul 13, 2022
aa995b8
updated example
DanailH Jul 13, 2022
37f8117
work on tests
DanailH Jul 13, 2022
e53ca5b
resolve conficts
DanailH Jul 26, 2022
4038f5d
PR comments
DanailH Jul 26, 2022
f880252
fix apis
DanailH Jul 26, 2022
a61e86c
resolve conflicts
DanailH Aug 2, 2022
c4a9343
polish the demo
DanailH Aug 2, 2022
cd87b6c
remove uneeded oage
DanailH Aug 3, 2022
4a31d2f
fix docs wording
DanailH Aug 3, 2022
fa8114f
Update packages/grid/x-data-grid-pro/src/hooks/features/lazyLoader/us…
DanailH Aug 3, 2022
6dd928b
Update packages/grid/x-data-grid/src/utils/utils.ts
DanailH Aug 3, 2022
875bd0c
working
DanailH Aug 3, 2022
877a117
optimise code
DanailH Aug 3, 2022
3fc84ff
Update docs/data/data-grid/row-updates/row-updates.md
DanailH Aug 22, 2022
42e11ee
Update docs/data/data-grid/row-updates/row-updates.md
DanailH Aug 22, 2022
aac485f
Update docs/data/data-grid/row-updates/row-updates.md
DanailH Aug 22, 2022
8906b83
PR comments
DanailH Aug 22, 2022
6ef5dc1
resolve conflicts
DanailH Aug 22, 2022
9bd6a08
Update packages/grid/x-data-grid/src/hooks/features/rows/useGridRows.ts
DanailH Aug 22, 2022
9dab887
optimize update rows as per PR comments
DanailH Aug 22, 2022
5b946e3
Merge branch 'feature/DataGridPro-lazy-loading' of github.com:DanailH…
DanailH Aug 22, 2022
e2661ce
Update packages/grid/x-data-grid-pro/src/tests/lazyLoader.DataGridPro…
DanailH Aug 22, 2022
fedb8c5
Update packages/grid/x-data-grid-pro/src/tests/lazyLoader.DataGridPro…
DanailH Aug 22, 2022
26a42ed
work
DanailH Aug 22, 2022
f0c9393
Merge branch 'feature/DataGridPro-lazy-loading' of github.com:DanailH…
DanailH Aug 22, 2022
09ad985
Update docs/data/data-grid/row-updates/row-updates.md
DanailH Aug 22, 2022
ead9b2a
Update docs/data/data-grid/row-updates/row-updates.md
DanailH Aug 22, 2022
0139c01
move docs info
DanailH Aug 22, 2022
57f07c9
Merge branch 'feature/DataGridPro-lazy-loading' of github.com:DanailH…
DanailH Aug 22, 2022
d890470
resolve conflicts
DanailH Aug 22, 2022
5bd7d0d
fix build
DanailH Aug 22, 2022
801b443
fix liter
DanailH Aug 22, 2022
1f1074e
simplify skeletonRowId
DanailH Aug 22, 2022
acdcb76
Update packages/grid/x-data-grid/src/hooks/features/rows/useGridRows.ts
DanailH Aug 22, 2022
c8f46e9
Update packages/grid/x-data-grid-pro/src/hooks/features/lazyLoader/us…
DanailH Aug 22, 2022
f55e7fe
infer return types
DanailH Aug 22, 2022
e1a8044
Merge branch 'feature/DataGridPro-lazy-loading' of github.com:DanailH…
DanailH Aug 22, 2022
1d845d0
simplify skeleton row check
DanailH Aug 22, 2022
f0aeea7
Update docs/data/data-grid/row-updates/row-updates.md
DanailH Aug 23, 2022
dc4e50b
revert changes
DanailH Aug 23, 2022
90e1903
resolve conflicts
DanailH Aug 23, 2022
61be031
Update docs/data/data-grid/row-updates/row-updates.md
DanailH Aug 24, 2022
7d5e427
Update packages/grid/x-data-grid/src/models/params/gridRenderedRowsIn…
DanailH Aug 24, 2022
d6b536f
Update packages/grid/x-data-grid/src/models/params/gridRenderedRowsIn…
DanailH Aug 24, 2022
0adb7ef
PR comments
DanailH Aug 24, 2022
65e522b
Merge branch 'feature/DataGridPro-lazy-loading' of github.com:DanailH…
DanailH Aug 24, 2022
9a88194
resolve conflicts
DanailH Aug 24, 2022
2dafe3a
fix build
DanailH Aug 24, 2022
d781e29
Update packages/grid/x-data-grid-pro/src/models/gridFetchRowsParams.ts
DanailH Aug 24, 2022
4491fb4
Update packages/grid/x-data-grid-pro/src/models/gridFetchRowsParams.ts
DanailH Aug 24, 2022
6cf0cf9
Update packages/grid/x-data-grid-pro/src/tests/lazyLoader.DataGridPro…
DanailH Aug 24, 2022
39ad75f
Update packages/grid/x-data-grid/src/hooks/features/rows/useGridRows.ts
DanailH Aug 24, 2022
c563079
feedback
DanailH Aug 24, 2022
fa846a6
conflicts
DanailH Aug 24, 2022
f381442
update demo and fix sorting/filtering bug
DanailH Aug 24, 2022
924cf66
work
DanailH Aug 24, 2022
74200f4
add new test case
oliviertassinari Aug 24, 2022
7512a8f
simplify demo
oliviertassinari Aug 24, 2022
3d12945
update the loading rows count
DanailH Aug 25, 2022
69b0595
get latest
DanailH Aug 25, 2022
17def05
rework replaceRows method
DanailH Aug 25, 2022
79eb868
add emply rows check
DanailH Aug 25, 2022
88ea46f
rework how scrolling works after filtring or sorting
DanailH Aug 25, 2022
3724946
update docs
DanailH Aug 25, 2022
38be38d
add unstbale_ prefix and include debounce in the demo
DanailH Aug 31, 2022
5a7905e
remove console.log
DanailH Aug 31, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions docs/data/data-grid/events/events.json
Original file line number Diff line number Diff line change
Expand Up @@ -204,6 +204,14 @@
"params": "GridEditRowsModel",
"event": "MuiEvent<{}>"
},
{
"projects": ["x-data-grid-pro", "x-data-grid-premium"],
"name": "fetchRows",
"description": "Fired when a new batch of rows is requested to be loaded. Called with a GridFetchRowsParams object.",
"params": "GridFetchRowsParams",
"event": "MuiEvent<{}>",
"componentProp": "onFetchRows"
},
{
"projects": ["x-data-grid", "x-data-grid-pro", "x-data-grid-premium"],
"name": "filterModelChange",
Expand Down Expand Up @@ -264,6 +272,13 @@
"event": "MuiEvent<{}>",
"componentProp": "onPreferencePanelOpen"
},
{
"projects": ["x-data-grid", "x-data-grid-pro", "x-data-grid-premium"],
"name": "renderedRowsIntervalChange",
"description": "Fired when the rendered rows index interval changes. Called with a GridRenderedRowsIntervalChangeParams object.",
"params": "GridRenderedRowsIntervalChangeParams",
"event": "MuiEvent<{}>"
},
{
"projects": ["x-data-grid", "x-data-grid-pro", "x-data-grid-premium"],
"name": "resize",
Expand Down
2 changes: 1 addition & 1 deletion docs/data/data-grid/row-updates/InfiniteLoadingGrid.js
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ export default function InfiniteLoadingGrid() {

React.useEffect(() => {
return () => {
mounted.current = false;
mounted.current = true;
};
}, []);

Expand Down
2 changes: 1 addition & 1 deletion docs/data/data-grid/row-updates/InfiniteLoadingGrid.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ export default function InfiniteLoadingGrid() {

React.useEffect(() => {
return () => {
mounted.current = false;
mounted.current = true;
};
}, []);

Expand Down
106 changes: 106 additions & 0 deletions docs/data/data-grid/row-updates/LazyLoadingGrid.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
import * as React from 'react';
import { debounce } from '@mui/material/utils';
import { DataGridPro, useGridApiRef } from '@mui/x-data-grid-pro';
import { createFakeServer, loadServerRows } from '@mui/x-data-grid-generator';

const DATASET_OPTION = {
dataSet: 'Employee',
rowLength: 10000,
};

const { columns, columnsWithDefaultColDef, useQuery } =
createFakeServer(DATASET_OPTION);

const emptyObject = {};

export default function LazyLoadingGrid() {
// dataServerSide simulates your database.
const { data: dataServerSide } = useQuery(emptyObject);

const apiRef = useGridApiRef();
const [initialRows, setInitialRows] = React.useState([]);
const [rowCount, setRowCount] = React.useState(0);

const fetchRow = React.useCallback(
async (params) => {
const serverRows = await loadServerRows(
dataServerSide,
{
filterModel: params.filterModel,
sortModel: params.sortModel,
},
{
minDelay: 300,
maxDelay: 800,
useCursorPagination: false,
},
columnsWithDefaultColDef,
);

return {
slice: serverRows.returnedRows.slice(
params.firstRowToRender,
params.lastRowToRender,
),
total: serverRows.returnedRows.length,
};
},
[dataServerSide],
);

// The initial fetch request of the viewport.
React.useEffect(() => {
if (dataServerSide.length === 0) {
return;
}

(async () => {
const { slice, total } = await fetchRow({
firstRowToRender: 0,
lastRowToRender: 10,
sortModel: [],
filterModel: {
items: [],
},
});

setInitialRows(slice);
setRowCount(total);
})();
}, [dataServerSide, fetchRow]);

// Fetch rows as they become visible in the viewport
const handleFetchRows = React.useCallback(
async (params) => {
const { slice, total } = await fetchRow(params);

apiRef.current.unstable_replaceRows(params.firstRowToRender, slice);
setRowCount(total);
},
[apiRef, fetchRow],
);

const debouncedHandleFetchRows = React.useMemo(
() => debounce(handleFetchRows, 200),
[handleFetchRows],
);

return (
<div style={{ height: 400, width: '100%' }}>
<DataGridPro
columns={columns}
rows={initialRows}
apiRef={apiRef}
hideFooterPagination
rowCount={rowCount}
sortingMode="server"
filterMode="server"
rowsLoadingMode="server"
onFetchRows={debouncedHandleFetchRows}
experimentalFeatures={{
lazyLoading: true,
}}
/>
</div>
);
}
114 changes: 114 additions & 0 deletions docs/data/data-grid/row-updates/LazyLoadingGrid.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
import * as React from 'react';
import { debounce } from '@mui/material/utils';
import {
DataGridPro,
GridFetchRowsParams,
useGridApiRef,
} from '@mui/x-data-grid-pro';
import {
createFakeServer,
loadServerRows,
UseDemoDataOptions,
} from '@mui/x-data-grid-generator';

const DATASET_OPTION: UseDemoDataOptions = {
dataSet: 'Employee',
rowLength: 10000,
};

const { columns, columnsWithDefaultColDef, useQuery } =
createFakeServer(DATASET_OPTION);

const emptyObject = {};

export default function LazyLoadingGrid() {
oliviertassinari marked this conversation as resolved.
Show resolved Hide resolved
// dataServerSide simulates your database.
const { data: dataServerSide } = useQuery(emptyObject);

const apiRef = useGridApiRef();
const [initialRows, setInitialRows] = React.useState<typeof dataServerSide>([]);
const [rowCount, setRowCount] = React.useState(0);

const fetchRow = React.useCallback(
async (params: GridFetchRowsParams) => {
const serverRows = await loadServerRows(
dataServerSide,
{
filterModel: params.filterModel,
sortModel: params.sortModel,
},
{
minDelay: 300,
maxDelay: 800,
useCursorPagination: false,
},
columnsWithDefaultColDef,
);

return {
slice: serverRows.returnedRows.slice(
params.firstRowToRender,
params.lastRowToRender,
),
total: serverRows.returnedRows.length,
};
},
[dataServerSide],
);

// The initial fetch request of the viewport.
React.useEffect(() => {
if (dataServerSide.length === 0) {
return;
}

(async () => {
const { slice, total } = await fetchRow({
firstRowToRender: 0,
lastRowToRender: 10,
sortModel: [],
filterModel: {
items: [],
},
});

setInitialRows(slice);
setRowCount(total);
})();
}, [dataServerSide, fetchRow]);

// Fetch rows as they become visible in the viewport
const handleFetchRows = React.useCallback(
async (params: GridFetchRowsParams) => {
const { slice, total } = await fetchRow(params);

apiRef.current.unstable_replaceRows(params.firstRowToRender, slice);
setRowCount(total);
},
[apiRef, fetchRow],
);

const debouncedHandleFetchRows = React.useMemo(
() => debounce(handleFetchRows, 200),
[handleFetchRows],
);

return (
<div style={{ height: 400, width: '100%' }}>
<DataGridPro
columns={columns}
rows={initialRows}
apiRef={apiRef}
hideFooterPagination
rowCount={rowCount}
sortingMode="server"
filterMode="server"
rowsLoadingMode="server"
onFetchRows={debouncedHandleFetchRows}
experimentalFeatures={{
lazyLoading: true,
}}
/>
</div>
);
}
14 changes: 14 additions & 0 deletions docs/data/data-grid/row-updates/LazyLoadingGrid.tsx.preview
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
<DataGridPro
columns={columns}
rows={initialRows}
apiRef={apiRef}
hideFooterPagination
rowCount={rowCount}
sortingMode="server"
filterMode="server"
rowsLoadingMode="server"
onFetchRows={debouncedHandleFetchRows}
experimentalFeatures={{
lazyLoading: true,
}}
/>
34 changes: 34 additions & 0 deletions docs/data/data-grid/row-updates/row-updates.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,40 @@ In addition, the area in which `onRowsScrollEnd` is called can be changed using

{{"demo": "InfiniteLoadingGrid.js", "bg": "inline", "disableAd": true}}

## Lazy loading [<span class="plan-pro"></span>](/x/introduction/licensing/#pro-plan)

DanailH marked this conversation as resolved.
Show resolved Hide resolved
:::warning
This feature is experimental, it needs to be explicitly activated using the `lazyLoading` experimental feature flag.

```tsx
<DataGridPro experimentalFeatures={{ lazyLoading: true }} {...otherProps} />
```

:::

Lazy Loading works like a pagination system, but instead of loading new rows based on pages, it loads them based on the viewport.
It loads new rows in chunks, as the user scrolls through the grid and reveals empty rows.

The data grid builds the vertical scroll as if all the rows are already there, and displays empty (skeleton) rows while loading the data. Only rows that are displayed get fetched.

To enable lazy loading, there are a few steps you need to follow:

First, set `rowsLoadingMode="server"`.
Then, set `rowCount` to reflect the number of available rows on the server.
Third, set a callback function on `onFetchRows` to load the data corresponding to the row indices passed within `GridFetchRowsParams`.
Finally, replace the empty rows with the newly fetched ones using `apiRef.current.replaceRows()` like in the demo below.

{{"demo": "LazyLoadingGrid.js", "bg": "inline", "disableAd": true}}

:::warning
The `onFetchRows` callback is called every time a new row is in the viewport, so when you scroll, you can easily send multiple requests to your backend. We recommend developers limit those by implementing debouncing.
:::

:::info
DanailH marked this conversation as resolved.
Show resolved Hide resolved
In order for filtering and sorting to work you need to set their modes to `server`.
You can find out more information about how to do that on the [server-side filter page](/x/react-data-grid/filtering/#server-side-filter) and on the [server-side sorting page](/x/react-data-grid/sorting/#server-side-sorting).
:::

## High frequency [<span class="plan-pro"></span>](/x/introduction/licensing/#pro-plan)

Whenever the rows are updated, the grid has to apply the sorting and filters. This can be a problem if you have high frequency updates. To maintain good performances, the grid allows to batch the updates and only apply them after a period of time. The `throttleRowsMs` prop can be used to define the frequency (in milliseconds) at which rows updates are applied.
Expand Down
7 changes: 6 additions & 1 deletion docs/pages/x/api/data-grid/data-grid-premium.json
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@
"experimentalFeatures": {
"type": {
"name": "shape",
"description": "{ aggregation?: bool, columnGrouping?: bool, newEditingApi?: bool, preventCommitWhileValidating?: bool, rowPinning?: bool, warnIfFocusStateIsNotSynced?: bool }"
"description": "{ aggregation?: bool, columnGrouping?: bool, lazyLoading?: bool, newEditingApi?: bool, preventCommitWhileValidating?: bool, rowPinning?: bool, warnIfFocusStateIsNotSynced?: bool }"
}
},
"filterMode": {
Expand Down Expand Up @@ -159,6 +159,7 @@
},
"onEditRowsModelChange": { "type": { "name": "func" } },
"onError": { "type": { "name": "func" } },
"onFetchRows": { "type": { "name": "func" } },
"onFilterModelChange": { "type": { "name": "func" } },
"onMenuClose": { "type": { "name": "func" } },
"onMenuOpen": { "type": { "name": "func" } },
Expand Down Expand Up @@ -205,6 +206,9 @@
"rowHeight": { "type": { "name": "number" }, "default": "52" },
"rowModesModel": { "type": { "name": "object" } },
"rowReordering": { "type": { "name": "bool" } },
"rowsLoadingMode": {
"type": { "name": "enum", "description": "'client'<br>&#124;&nbsp;'server'" }
},
"rowSpacingType": {
"type": { "name": "enum", "description": "'border'<br>&#124;&nbsp;'margin'" },
"default": "\"margin\""
Expand Down Expand Up @@ -268,6 +272,7 @@
"cell",
"cellContent",
"cellCheckbox",
"cellSkeleton",
"checkboxInput",
"columnHeader--alignCenter",
"columnHeader--alignLeft",
Expand Down
Loading