[docs-infra] Improve Algolia internal search results on mui.com

[eps1lon]

Listing various points that could be improved for searching our docs. Also documenting unhelpful search results. It's not about implementing all of those but exploring how viable they are or if they have better alternatives.

Opportunities

• 1. Be able to search inside the content of the page, not just headers.
• 2. Broken UI on blog post, e.g. https://mui.com/blog/v6-beta-pickers/. The style loading logic is broken. [docs-infra] The icons in the search dialog appear abnormally #33731
• 3. Easy switch between projects, e.g. https://www.mux.com/blog/the-building-blocks-of-great-docs [docs-infra] Add capacity to filter the search scope #41541

or https://twitter.com/delba_oliveira/status/1664323492077256704

• 4. The search bar in the header blinks when changing pages. This shouldn't happen.
• 5. Remove the #main-content noise. Reproduction: search for Alert
  
  You land on https://mui.com/material-ui/react-alert/#main-content. There are tons of links like this in Stack Overflow, we can do better, the UX is confusing, no, it doesn't link an h2, it's the h1.
  Fixed by adding:

$("#main-content, #__next").removeAttr("id");

In the crawler logic (https://crawler.algolia.com/admin/crawlers/739c29c8-99ea-4945-bd27-17a1df391902/configuration/edit).

• 6. Implement OpenSearch [docs-infra] Implement search engine in v5 documentation page #29445

• 7. Index MUI X. e.g. https://mui.com/x/introduction/licensing/ and related pages. It would solve [docs] Instruction to deal with invalid license message mui-x#5074 (comment)

• 8. Index MUI System e.g. https://mui.com/system/getting-started/overview/ and related pages. It would solve https://www.algolia.com/apps/TZGZ85B9TB/analytics/no-results/material-ui?from=2022-09-14&to=2022-09-20.

• 9. Move the product attribute from the crawler configuration https://crawler.algolia.com/admin/crawlers/739c29c8-99ea-4945-bd27-17a1df391902/configuration/edit to the page, e.g.

  material-ui/docs/data/joy/components/alert/alert.md

  Line 2 in 29fd59e

  product: joy-ui

  . It would be simpler. For example, from a docs versioning perspective, it's painful to have to come to Algolia's Crawler to make a change to have things keep working.

• 10. Index MUI Toolpad, e.g. https://mui.com/toolpad/getting-started/overview/

• 11. Fix moving scrollbar when opening the search (you can test on Windows)

• 12. Migrate the UI to Add Spotlight/Omnibar component mui-x#13932

• 13. Duplicate lang property https://www.algolia.com/apps/TZGZ85B9TB/explorer/browse/material-ui?page=1&searchMode=objectID

  

  Likely a conflict between

  material-ui/docs/src/modules/components/Head.tsx

  Line 58 in 29fd59e

  <meta name="docsearch:language" content={userLanguage} />

  and https://docsearch.algolia.com/docs/record-extractor#indexing-content-for-faceting

• 14. Move popularity and pageRank configuration from the Crawler UI to the markdown pages.

• 15. Use analyticsTags to so that each product team can filter the analytics in the Algolia admin

  

  Fixed in [docs-infra] Add analyticsTags to Algolia #37600

• 16. Implement click analytics to measure the impact of search ranking changes Click and Conversion Analytics algolia/docsearch#1045

Fix unhelpful results

• "FilledInput" (top 5 results link to /api/filled-input page)
• "material tabs"

[oliviertassinari]

I wish Algolia docsearch was implementing the analytics conversion API. Quantitative data could help. At least, we could measure the global impact of our changes. For instance, we could consider a successful conversion when staying more than 30s? on the result page. Or maybe we could start simpler and only implement the click event on a suggestion.