Discover Rewrite Proposal -- Draft

[ananzh]

Discover overview

Discover is an internal plugin in OpenSearch Dashboards core. There are three major functions:

• Select data for your exploration, and then set a time range for that data, search it with the OpenSearch Dashboards Query Language, and filter the results.
• Explore the details of your data, view individual documents, and create tables that summarize the contents of the data.
• Present your findings in a visualization.

Importance of Discover Rewrite

Discover is one important and irreplaceable piece among three visualization blocks in the OpenSearch Dashboards.

• Long-term and widely used component: Discover is one of the oldest visualization tool to explore the details of data. There are over 500+ kibana issues before we fork which all shows that customers are used to visualize data through discover.
• A connection of three major visualization blocks: The three blocks, dashboards, visualize and discover are not isolated blocks. They are actually connected to each other through discover. When customer saves search/query in discover, discover will create a saved object with type savesearch. This saved search of doc view from discover can be added to dashboards directly. It can also be used to create different visualization types similar to index pattern.
• A source to understand customer: Anything with long term existence must have a good reason. From Discover, we could understand more on how customer visualize data. With rewrite, customer won't feel any major difference in usage.

Discover components

1. Top Nav[1]: To search for data within an index pattern. This allows customer to search and filter the index pattern. It also allows the user to save as a savedsearch object, which is a saved object type for discover. This savedsearch object can be used to create visualization directly, similar to index pattern. It could also be added to dashboards directly and display doc views.
2. Histogram Visualization: It is used to visualize data count for given search date in histogram visualization. It allows manually refresh when date is changed or auto-refresh to catch new data.
3. Table Visualization: It is used to explore data details. When a document is clicked, it will show doc views in JSON/Table format.
4. Data Panel [1]: A panel for accessing the index pattern and its available fields. The data panel includes an index pattern selector, a search bar to search field in the index pattern, Selected fields to show all the selected fields, and a field summary of each field, when hovered over or clicked, will show us a summary of the data in the field.

Rewrite effort evaluation

We have done researches on almost all the above components and right now team has already built a fundamental knowledge and experience of rewriting these components.

1. Top Nav
   Vis Builder design and implement a similar top navigation [Design] Drag & Drop Visualization CX #896
   Saved objects are well researched and implemented [D&D] Research on how plugins create and store reference objects #2205
2. Table Visualization:
   Table visualization is re-write in react via Datagrid [DESIGN][Table] new table visualization plugin #2187
   We could implement a better table vis via Datagrid and doc view via OuiFlex.
3. Data Panel
   We have a similar data panel in vis builder
   Field summary popover [Vis Builder] Add field summary popovers #2682

For Histogram Visualization, it seems not related closely to the main functionality of discover and not necessary. Could either rewrite or remove.

Another thing I would to point out is that since Discover is an old visualization tool, we have a rich function test set (test/functional/apps/discover) to make sure all the functionalities are captured in the new Discover.

Reference

[1] #896

[kavilla]

Not sure if worth calling out but I know often it is helpful when an index pattern gets docs and the fields mark it as unknowned since the field has not been mapped in the index pattern.

[ananzh]

Hmmm @kavilla could you give some explanations on the above comment?

[ananzh]

We have decided to replace Discover. Close the issue.