Show "rich" meta data about remote URL when adding URLs as links

[getdave]

What problem does this address?

Currently when adding a link in the editor, you see a series of results based on whatever you type into the search field.

These are different depending on what text you enter and the component tries to "intelligently" discern whether you are:

1. Directly entering a URL
2. Searching for a WordPress entity (eg: Post, Page, Category...etc).

When you enter a direct URL such as www.wordpress.org (note the www. is required) then the UI simply displays a single result which is a facsimile of what you entered:

This is ok, but it could be better and provide better visual confirmation.

What is your proposed solution?

I'm proposing we make both:

1. the search results
2. the selected link details

...a little "richer" by enhancing them with meta data from the remote URL such as:

• <title> tag contents.
• favicon/site icon.
• Contents of <meta name="description" /> tag.

Here's an example of that

Screen.Capture.on.2021-05-04.at.15-38-05.mov

This would provide a visual confirmation to the user that they have entered a valid URL and that it is the one they are expecting.

Also here is a screenshot showing where we have opportunity to make the selected link details richer:

...for example of how this could look here is what Google Docs shows:

I see this purely as a progressive enhancement - ie: if we cannot enhance the given entry then we simply display the experience as it is now. However, if we do manage to get data then we can enhance the search result.

We already have the REST API endpoint to get the data (I'm currently working on making it return more data) so it's about wiring things together and getting a good UX design.

Todo

• Enhance link control UI with remote URL data #31464
• Add REST API endpoint to retrieve data about a remote URL.
• Improve REST API endpoint parsing to retrieve more data and improve stability.

[getdave]

@javierarce I think this could be a really nice enhancement. I wonder would you be able to help with the Design UX here?

[skorasaurus]

ref #27762

[javierarce]

@getdave I love the idea! I'll help you with the design.

[getdave]

Screen.Capture.on.2021-05-04.at.16-49-16.mov

Here's an example of use fetching details for the link preview to provide a richer experience.

Obviously, it has some glitches, but essentially you see the url (which is what we store in the Gutenberg data) whilst the request goes out to fetch the "rich" data. Once that request completes then the UI updates to show the "rich data".

I'm experimenting in #31464

[javierarce]

Ok, here's the first iteration of this feature's design.

This is the basic add flow where the user searches for an item and we show a list of results. If the text happens to be an URL, we fetch the metadata and render a card (at this point, I wonder if it would be OK to fetch the metadata when the search term doesn't begin with www.)

The most notable changes I'm introducing are:

• In the current design, the items from the autocomplete are contained inside a card whose background changes on hover. I think we could get rid of the effect and highlight the titles instead (that said, I've explored a couple of variations here)
• I've also removed the icon that indicates that a link will get open on a new page. I think this would add a lot of noise in a section containing an avatar and the edit button.

Here are a table with the possible variations depending on what kind of data is available:

I'm a bit unsure about the Press ENTER to add this link help text. I think it works fine when there isn't an image, but when there's one, it gets kind of sandwiched between the image and the title.

This Figma prototype lets you play with the different types of cards.

Screen.Recording.2021-05-05.at.17.46.27.mov

[getdave]

@javierarce This is really great - thanks so much for this work and insight.

I like the way the different card options allow for a progressively enhanced approach with increasing levels of fidelity. This is very important as it's not guaranteed that we can fetch metadata in every case.

For a v1 I'm going to propose we go with Favicon + Title + Description. This should be easy enough to parse by enhancing the existing endpoint. I'm thinking grabbing a suitable image will be more difficult so that could a v2 enhancement - I really like the way it looks. Do you agree?

I'll start by enhancing the existing REST API endpoint and then try to pull some of that work into #31464 which is a PoC PR which already does some of what you are planning here.

[javierarce]

For a v1 I'm going to propose we go with Favicon + Title + Description. This should be easy enough to parse by enhancing the existing endpoint. I'm thinking grabbing a suitable image will be more difficult so that could a v2 enhancement -

That's perfect!

[getdave]

I'm currently working on improving the endpoint parsing capabilities in #31763