Introduce Data providers

[Janpot]

Screen.Recording.2023-09-25.at.14.40.43.mov

To do:

• Add a switch in the property controls to switch between row mode and dataprovider mode
• documentation
• test
• reset cursor when pageSize changes
• ?

Documentation:
https://deploy-preview-2644--mui-toolpad-docs.netlify.app/toolpad/concepts/data-providers/
https://deploy-preview-2644--mui-toolpad-docs.netlify.app/toolpad/reference/api/create-data-provider/

[apedroferreira]

Maybe i'm not super familiar with cursor pagination, but it isn't completely obvious to me right away how to use this cursor pagination.
is the cursor a number? Or can I use the id of a record?
The prisma documentation on this kind of thing is very clear, maybe we could use it as inspiration https://www.prisma.io/docs/concepts/components/prisma-client/pagination.

[apedroferreira]

Oh, also the way Prisma does it kind of just works and we don't have to explicitly choose a pagination model. Not sure if we could do something similar?

[apedroferreira]

Nevermind I understand this is quite different from Prisma now as it integrates very directly with the Data Grid, these were just some quick ideas while going over the docs and trying to use the feature in a basic way. Feel free to ignore if they don't make sense or don't provide any insights, I'm sure you looked at these problems deeper than me :D

[Janpot]

I've added another paragraph explaining the value of cursor in this pagination model.

Oh, also the way Prisma does it kind of just works and we don't have to explicitly choose a pagination model. Not sure if we could do something similar?

In prisma you don't need to explicitly specify a pagination model because it doesn't need to signal to consumers how it is to be called. The Toolpad data provider needs to signal to Toolpad how it's to be fetched. Supplying this option configures the interfaces in getRecords for the right pagination model.

[apedroferreira]

Looks good and it's easy to use, really great feature! Here's some feedback I collected while trying it out, just trying to give as much feedback as possible:

• I set up some static data for records but forgot to include id strings, so an error message showed in the Data Grid telling me to do that. Once I included the ids and saved the file, the Data Grid did not update without the error until I refreshed the page.
• Maybe we could leave the cursor choice for later if possible instead of having it in the modal right away, and start with index pagination automatically? But this could depend on many things, it's just a thought to reduce initial friction/confusion a bit if someone's trying the feature for the first time.
• If I'm using the data grid for the first time, I notice these options and I try to create a data provider, how can I quickly get to the documentation so I can understand all the options I have? As this can be a complex feature, maybe we can make this easier. Maybe we can have a comment with a link to the docs in the data provider template (and eventually more helpful comments in general).

[apedroferreira]

Why not just check this inside the errorMessage function itself?

[Janpot]

nameExists is a boolean and existingNames is a Set. Therefore nameExists is automatically stable, existingNames depends on the consumer to provide a stable value. .has is a fast operation. Not sure which is better, but don't believe it matters much. I can put it inside as well.

[apedroferreira]

Got it, it's not important so we can keep it, I think it's fine either way.

[apedroferreira]

Not sure if it would be better to pass a validation function instead, anyway not a big deal.

[Janpot]

• I set up some static data for records but forgot to include id strings, so an error message showed in the Data Grid telling me to do that. Once I included the ids and saved the file, the Data Grid did not update without the error until I refreshed the page.

I added an error boundary to the grid. Also adding ids automatically when data provider is used, just the way it is done when rows prop is used. This also uncovered some problems around column inference in the columns editor which had the same root cause

• Maybe we could leave the cursor choice for later if possible instead of having it in the modal right away, and start with index pagination automatically? But this could depend on many things, it's just a thought to reduce initial friction/confusion a bit if someone's trying the feature for the first time.

The choice of pagination mode is made in the code only. It's not an option that can be easily changed from the UI. This initializes the code with the right pagination mode. Initially this wasn't part of data providers but was suggested in the last grooming call.

• If I'm using the data grid for the first time, I notice these options and I try to create a data provider, how can I quickly get to the documentation so I can understand all the options I have? As this can be a complex feature, maybe we can make this easier. Maybe we can have a comment with a link to the docs in the data provider template (and eventually more helpful comments in general).

• I added a helper text to the pagination mode input with a link to the documentation

  

• vscode also displays a tooltip with links to the docs when you hover createDataProvider: https://github.com/mui/mui-toolpad/pull/2644/files#diff-1df7e8b1e0c8176606fffa17934efa350f0483e6a994b40f5fba2a3c2b61ce23R135

• also added a link to the docs in the generated code as a comment

[apedroferreira]

The changes look great! Thanks for addressing the feedback.