Improve experience on dashboard when visualizations do not use index patterns created in Kibana

[bhavyarm]

Kibana version: 6.3.0 BC6

Browser version: chrome latest

Browser OS version: OS X

Original install method (e.g. download page, yum, from source, etc.): from staging

Describe the bug: If user has tsvb, markdown, input control, and vega vizs only on a dashboard( and they are not on default Kibana index pattern) and shares the iFrame - the add filter on them becomes useless(? - not sure if I am using the correct word here). Because its populated with fields from default index pattern in Kibana.

Steps to reproduce:

1. Create a dashboard with only TSVB, markdown, input control, and/or vega vizs
2. share the dashboard with an iFrame link (That step isn't even required)
3. Click on add filter - it has only fields from the default index pattern of Kibana

Expected behavior:
We need to either disable add filter or show a message to the user?

Screenshots (if relevant):

[bhavyarm]

cc @timroes / @stacey-gammon this is both bug and enhancement :)

[stacey-gammon]

Alright so I think this is more an issue with the experimental visualizations than with dashboard. They aren't giving dashboard the index patterns they use because they don't use index patterns created in Kibana.

Dashboard passes those index patterns to the filter bar.

Essentially any visualization that doesn't supply index patterns can't take advantage of the drop downs in the filter bar.

I tagged a bunch more groups since this is more of a problem of a global problem. I also don't have a good answer for how this should be handled.

[timroes]

Indeed that's a global problem and might even become more tricky over time. Currently dashboard expects every visualization to pass it's used index pattern (I think actually only in singular). The mentioned visualizations all allow specifying multiple index patterns. So first we need to make sure that every embeddable can return multiple index patterns.

The next problem is, that TSVB, Vega and Timelion are not using index patterns at all, and bypass them, so basically we would need to make them load an appropriate index pattern (if one exist) for the index names they use in their visualization.

Last but not least, once we switched over to rendering using the Canvas pipeline that issue will apply to ALL visualization, since all of them could potentially use multiple index patterns in their expression. Also just because an index pattern is used in the pipeline, doesn't mean that its'a actually used for rendering the chart, e.g. you could load data from one index pattern just to calculate some values, e.g. for axis extends. Also from the outside we have no idea what parameter of a function inside the expression might actually be an index pattern. So we have basically two choices: Make expressions need to declare what parameters are index pattern or have some utility class, more or less blindly parsing the pipeline and that utility has hard coded logic for some of the functions we know, to extract the index pattern from it (which of course means, the behavior would break again, if you provide your own expression functions).

I find neither of those options ideal, we either have only partial working hacky solution or we need to make the expression function API more complex.

I think we need some time to discuss that hole feature all together.

[timroes]

Update from 7.6.1:

This is no longer true for TSVB, since it now communicates it's index pattern correctly. Markdown and input controls are anyway not affected by filters, so it really doesn't make sense applying any filters to them anyway. Vega is still a problem, since we don't have any idea which index patterns are used, since the user provide their query themselves, we cannot communicate that to the dashboard to make that index pattern available.

The only really good suggestion I'd see right now, is if the embeddable layer maybe supports a way not to communicate your index pattern, but also a way to communicate, that you have no idea about the used index pattern, in which case the filter bar could potentially show the index pattern selection in the filter editor with all index patterns available. I am really not sure if that is better, since that would mean as soon as you place a Vega chart on a dashboard, you must always select the index pattern in the filter editor (and we know users can potentially have a lot of them). And you would suddenly also need to do that, even if Vega is actually only using that one index pattern, that maybe all other visualizations use too.

@stacey-gammon do you have any good new ideas for that?