[Lens] More intuitive sorting experience

[threat-punter]

Describe the feature:
Sorting results using Top values in Lens orders results by count by default instead of alphabetically.

Describe a specific use case for the feature:

We ran an internal capture the flag (CTF) event today and the expected behavior from several users was that choosing to sort results by Top values would order the results by count by default, not alphabetically. This seemed non-intuitive for participants.

[mbondyra]

Interesting, that's the default behaviour for visualize too. I think sorting by count allows a user to quickly see what's the distribution of the items in top values. What do you think @AlonaNadler @cchaos ?

Here's the screen from visualize:

[AlonaNadler]

I think by default we should order by the metric used in the chart descending
However, it should be easy to understand and change what the chart is sorted by.
I think maybe since we have 3 drop downs one after the other it hard to distinguish what the chart is sorted by

Aggregation and field are connected together, the aggregation is applied on the field. Maybe there is a way to show it visually in a more clear way.
Another option is to call out the order in the preview somehow

@cchaos @MichaelMarcialis any ideas

[elasticmachine]

Pinging @elastic/kibana-app (Team:KibanaApp)

[MichaelMarcialis]

I think by default we should order by the metric used in the chart descending
However, it should be easy to understand and change what the chart is sorted by.
I think maybe since we have 3 drop downs one after the other it hard to distinguish what the chart is sorted by

Aggregation and field are connected together, the aggregation is applied on the field. Maybe there is a way to show it visually in a more clear way.
Another option is to call out the order in the preview somehow

@cchaos @MichaelMarcialis any ideas

Apologies for the delayed reply; I was OOO last week. I agree that the default should be ordering by metric/count descending.

@AlonaNadler, regarding your question, are you speaking specifically about the Visualize "X-axis" accordion in the "Buckets" panel? If so, I would agree that the hierarchy for this section could be improved.

Perhaps the simplest solution would be to provide an additional level of headings to logically group form inputs. For example, "Order by," "Order," "Size," etc. could appear under an "Appearance" heading. "Aggregation" and "Field" could also be grouped under another logical heading. Also having the form input label text match the wording of Lens' inputs might be a nice, easy improvement as well. Happy to mock something up if need be. Let me know.

[wylieconlon]

I think there are two threads to follow up on based on what @AlonaNadler wrote previously:

1. Should we change the sort order to sort by metric when you delete the Y axis, and then add a new Y axis? We can change the sort order when adding/removing columns. It sounds like this is what Alona expected, and it's a small change to do. I think we should do this for fix-it-week:

I think by default we should order by the metric used in the chart descending

2. How will users know what the sort order is? I'm not sure what our options are, but maybe @MichaelMarcialis has a proposal that we could take a look at:

it should be easy to understand and change what the chart is sorted by.

[MichaelMarcialis]

2. How will users know what the sort order is? I'm not sure what our options are, but maybe @MichaelMarcialis has a proposal that we could take a look at:

I still believe the best way to make the ordering/sorting controls more easily findable to users would be to improve the general hierarchy of the form they are contained within (for example, we could divide the form into sections with appropriate headings, including a section for "Sorting"). If we're specifically talking about Lens in this case (and not the older visualization interfaces), I already have a task on my to-do list to propose improvements to the hierarchy of all Lens configuration flyouts, though it is currently the lowest priority.

If the priority of this task should be reconsidered, please let me know. Thanks!

[wylieconlon]

@MichaelMarcialis we've added an item to the weekly agenda to talk through any quick improvements we can make, because it seems like there is a lot of confusion with the current feature. My attempt to clarify the confusion is to rename the features:

• "Top values" could use the word "rank" instead of "sort" because we are excluding values based on rank. Top 5, top 10, etc.
• The sorting used in the data table is what most users expect when they see sorting. It's taking all available values and changing their order, without excluding any.

[stratoula]

I think this has been addressed in recent kibana versions