[tools] Add new page for Base UI npm downloads KPIs

[mnajdova]

Added new page for Base UI's npm KPIs.

[oliviertassinari]

Sweet 👍

[oliviertassinari]

Oh nice, it looks almost like the logic in https://docs.google.com/spreadsheets/d/1FHxZ456e1t_MNJZg9wrq7pLRFzknZ6jMBhFuW1j1Q8o/edit#gid=0 used to fetch the data (importdownload()).

function importdownload(start = '2023-01-01', end = '2023-06-01', packagesRange = [['react-dom', '@mui/base']]) {
  const packages = packagesRange[0].filter((cell) => cell !== '');
  console.log('URL', `https://npm-stat.com/api/download-counts?${addQuery({
    package: packages
  })}&from=${start}&until=${end}`);

  var jsondata = UrlFetchApp.fetch(`https://npm-stat.com/api/download-counts?${addQuery({
    package: packages
  })}&from=${start}&until=${end}`);

  let npmStateResponse = JSON.parse(jsondata.getContentText());

  const months = {};

  Object.keys(npmStateResponse).forEach((package) => {
    const downloads = npmStateResponse[package];

    Object.keys(downloads).forEach((date) => {
      const month = date.substring(0, 7);

      months[month] = months[month] || {};
      months[month][package] = months[month][package] || 0;
      months[month][package] += downloads[date];
    });
  });

  const output = [['month', ...packages]];

  Object.keys(months).sort().forEach((month) => {
    output.push([month, ...packages.map((package) => {
      if (package === 'react-dom') {
        return months[month][package];
      }
      return months[month][package] / months[month]['react-dom'];
    })]);
  });

  console.log('output', output);
  return output;
}

[mnajdova]

Yep, looks like the same thing 👍 I was adding to add similar page for all products

[oliviertassinari]

A side note, in https://docs.google.com/spreadsheets/d/1FHxZ456e1t_MNJZg9wrq7pLRFzknZ6jMBhFuW1j1Q8o/edit#gid=590909841, I also subtract with @mui/joy, underestimating the actual downloads of Base UI. Soon or later, we can move to @mui/core-downloads-tracker

[mnajdova]

I will leave it as is for now

[oliviertassinari]

Yeah, agree, makes more sense.

[oliviertassinari]

This feels like a component that would be better in the Toolpad editor, getting us a bit closer to the Google Sheet experience.

[mnajdova]

I don't understand, what do you mean in the Toolpad editor?

[Janpot]

I'm interested in learning about why you went for a code component to implement this table rather than use the table that is built in Toolpad. Looking at the code it seems like you use the code component mainly to preprocess the incoming data. At a glance, the problems to solve for us are:

• Computed columns
  Adding columns mapped from the raw rows, but statically known. e.g. "date" and "ratio". I believe we could solve this either by providing a low-code way of remapping data, or by supporting computed columns
• Dynamically created columns
  Creating new columns dynamically based on incoming data. e.g. adding a column for each package. It's not clear to me yet how we can support this comprehensively, would need to benchmark other tools for ideas
• reuse mapped data
  This is a problem I don't see directly here, but we need to think about how we can support e.g. showing a graph of the filtered data of the datagrid. Maybe we can think about providing a myGrid.filteredRows or something that other components can bind to. Not fully convinced about this yet as it tightly couples components to each other. (What if I want to delete/replace the datagrid, does the chart become useless? Do I have to redo all data mangling?)
  Perhaps we should offer an option to circumvent the static column definition and always infer from the incoming data?

[mnajdova]

Ah I see the question, initially I was structuring the data in the component itself, but then moved this logic in a serverless function, I will try to do this change and report back if I miss a feature.

[mnajdova]

I've updated the app to use the DataGrid component directly. The only feature I was missing (or didn't know how to do) is rename the title of the columns

[oliviertassinari]

Soon https://master--material-ui-x.netlify.app/x/react-charts/lines/#data-format 😁

Actually, the API would be different, it wouldn't work like recharts

<LineChart width={600} height={300} data={data}>

[mnajdova]

Suggested change

	const getColumns = (packages) => ([
	{ field: 'date', headerName: 'Month', width: 150 },
	...packages.map(packageName => ({
	field: packageName, headerName: packageName, width: packageName === '@radix-ui/react-primitive' || packageName === '@headlessui/react' ? 170 : 120
	})),
	{field: 'ratio', headerName: "Base UI marketshare", width: 150}
	]);
	const getColumns = (packages) => ([
	{ field: 'date', headerName: 'Month', width: 150 },
	...packages.map(packageName => ({
	field: packageName, headerName: packageName, width: 150
	})),
	{field: 'ratio', headerName: "Base UI marketshare", width: 150}
	]);

cc @apedroferreira this is how you can reproduce the issue with the table overflowing.

[Janpot]

Nice!

On Toolpad side, I think feature-wise there could be focus on

• The stats component and the chart component seem to be the two main reasons why internally we keep creating custom components
• select/autocomplete options, datagrid rows, list data,... I always see big expressions to filter/map query data into lists of objects.

[mnajdova]

@Janpot I didn't see that you approved the PR and in the mean time I added one more page. Could you do a quick check please! Thanks!

Here is a screenshot:

I've reused the existing NPM chart component. It took minutes to add this new page, I am pretty happy with it

[mnajdova]

I am merging, next I will be adding some dashboards using the Google analytics API.