Tagging

[w33ble]

Describe the feature:

• Static tags
  • elements
  • functions
  • templates
• 2nd phase is being able to tag any saved object, eg workpads, template elements

Fill me in with more

[elasticmachine]

Pinging @elastic/kibana-canvas

[rashidkpc]

@ryankeairns we'll need UI for anywhere we can display tags, eg:

• Workpad loader
• Element chooser
• Autocomplete

[elasticmachine]

Pinging @elastic/kibana-design

[ryankeairns]

For future reference, mockups of tagging for workpad templates can be found here - #23696

The related PR contains static tagging and tag filtering for workpad templates - #23966

If you don't want to dig, here are a couple of mockups related to the aforementioned issue and PR:

Filter workpads by tag (phase 1)

Manage workpad tags (future phase)

[ryankeairns]

Here's a link to a basic prototype that demonstrates tagging/filtering on elements and autocomplete:
https://marvelapp.com/5ec5hb5/screen/50860013

• In the Marvel link, you can click the Tag dropdown, the '+3', or close the modal to view the autocomplete mockup. For the latter, I played around with showing tag indicators in the lefthand side of the autocomplete panel, but its gets busy quickly considering each function may have more than one tag. Given that, I've opted for a simpler approach where any recommended functions appear first (these could be tag-based recommendations or canvas function/pipe driven). The actual tags would appear in the function documentation near the bottom of the righthand panel.

• Workpad/template tagging mockups are linked in my previous comment. We've also discussed tagging for element templates, but I've not yet fleshed out that feature design-wise. I'm imagining that it will work similarly to the tags in the 'Add element' modal (presuming element templates live there as well).

Here are a few screenshots from the above link:

[stacey-gammon]

I'd like to summarize some offline discussions with @cqliu1 and @clintandrewhall, pose some questions and make some proposals.

Summary of Background and Original Implementation Plan

Please anyone correct me if I am wrong, but this is my understanding:

The original implementation and UX plan for Canvas Tagging was to have tags on Saved Objects (workpads) and registered plugins (functions, elements and templates). Depending on where the plugin type was exposed in the UI, it may appear differently (workpad and template tags in a table, function tags in auto complete, element tags in the widget selection). These tags would not be user editable, they would be developer defined, and mostly static (could not be dynamically added to/removed from any plugins) but not entirely (could be removed from workpads just by editing the objects).

Conflicts with Kibana wide Saved Object Tags

The above implementation would create some strange interactions when Saved Object tags comes along. We would have to implement a migration plan to merge the two, which no one has yet come up with, and does not sound straightforward. Reasons why I suspect a migration would be complicated:

• Canvas tags are not stored in the Kibana index and the migration runs on the index - it's unclear if migrations would be able to access the plugin registry at the time migrations takes place in order to create a new saved object for each existing tag.
• It creates a strange interaction with tags on non-saved objects. If the same tag is mapped to a workpad and a function, does the tag become a saved object, but continue to exist as a duplicate static canvas tag?

Just a few questions that immediately come to mind for a migration step to highlight why I find this path not straightforward.

Implementing canvas tags on canvas saved objects would almost be entirely tech debt as soon as Saved Object tags comes along. It comes with risk, and developer, and QA, overhead in order to transition to a single implementation.

Proposal: Do not implement a canvas only tags solution on canvas saved objects

*Note: * This will probably come up so I will just come out and say that saved object tagging is not on a roadmap yet but the platform team knows it is important and are staffing up. There is also the option for a canvas team member to team up with a platform team member to help balance priorities and push it forward.

Static "tags", or "categories"

During our discussions, this brought up another possibility of distinguishing between the two types of tags. Even when Saved Object Tags comes along, how will that help users discover different types of plugins, such as functions, elements and templates?

What if we moved forward with a canvas only solution for tagging plugins (functions, elements, templates), but not saved objects (workpads)?

The following questions arise with this option:

1. What about objects that are represented via plugins today, but should be migrated to be Saved Objects?

Anything that is a Saved Object gets security, is user editable, is migrateable, and can be imported and exported, all using existing functionality. Templates are a great candidate for a Saved Objects. If we add the ability to tag templates now, but later convert them into saved objects, we are left again with a migration step.

Proposal: Don't display any UI for tags on Templates, since we know they will likely become saved objects.

2. If we go with the above proposal, that brings up the question of what will eventually be a saved object and what won't?

At first, it seemed pretty straightforward that functions will be plugins. But digging a little deeper... I'm not so sure. What if a developer wants to create a function that should be secured and only allowed for usage in a given space, or for their own usage (maybe the function expects data that only certain users will have access to)? What if we want to allow users to dynamically write javascript and create and manage their own pipeline functions, similar to how Google Spreadsheets has a script editor? All of a sudden this isn't so obvious.

Same with Elements. We are discussing element grouping and templating. These should probably be Saved Objects since users could create elements with sensitive information in them and "Save as a template", and want to have full control over who can re-use that template. If we are implementing templates as Saved Objects, it makes sense for Elements to also be Saved Objects, as simple, baseline templates.

Question: Is the benefit of displaying tags on functions and elements worth the complexity this will possibly bring down the line? Lets also consider...

3. Potential UX confusion when there are editable Saved Object Tags and non-saved object tags. @ryankeairns may be able to work some magic here, but I do not have a good proposal to solve this.

Consider this hypothetical situation in a world with Saved Object Tags: A user creates a tag called Marketing, makes it green, and adds it to some dashboard, visualizations, canvas workpads, etc. A developer writes an element and gives it a tag of Marketing and makes it blue. The original user goes in to create an element on canvas and notices that there is a function is tagged with Marketing but it's blue, not green... weird. They go into tag management and only see a green Marketing tag. They rename the Marketing tag, it does not propagate to the function "tag".

If we have a global search bar that is searchable using tags, a user might wonder why their saved objects showed up but not their canvas functions, or elements, that have the same tag.

The more similar the UIs are, the more confusing this would be. Currently elements are show in a grid view, but what if we switch to table view? That would look very similar to a saved object table with a tags column and a tags filter. A different name might be able to help, like "category", but I don't know if it would help enough.

Also consider the possible subsequent requests that would come along. That user says "I want to be able to edit my element tags, dynamically add and assign new ones?". It's a whole different system so would require a whole different implementation.

The more things that are Saved Objects, and the more we can re-use all the functionality built on top of it (e.g. exporting/importing elements).

My personal feeling is that we should punt on this feature because there isn't a lot of time till 6.7 and there are a lot of questions here. I don't think we should rush it unless we are confident we have the right path forward. Even if we had the right path forward, stability and performance, especially in the cloud environment, are top priority, more than any feature. Do we have enough time to tackle both and be confident we are implementing it correctly?

I'm interested to hear the team's feedback on all of these points.

[alexfrancoeur]

Before reading through the specifics here, I wanted to state my thoughts first. The way I understand it, Canvas workpads are saved objects (at least through the API). Personally, I think we should strive towards a consistent import/export mechanism and organizational capabilities across Kibana. While I love the fact that you can import / export workpads from within the UI and think we should introduce similar functionality elsewhere, I would hope that in the near future, Canvas workpads are also in the saved objects UI. We've made any saved maps created with the new Maps application saved objects and, when introduced, I would expect tags to be utilized here as well.

In the short term, I'm fine with have a more siloed approach for Canvas but ideally, I as a user would be able to tag a Canvas workpad, dashboard and a handful of visualizations with the same framework. This could not only give me an overview of all the saved objects with a tag, but allow for easy export and the ability to quickly "move out" my objects to a personal or team space.

Additionally, it's worth noting that "tags" are already implemented in at least two different areas within the Kibana UI.

Beats Central Management

Machine Learning Jobs

What if we moved forward with a canvas only solution for tagging plugins (functions, elements, templates), but not saved objects (workpads)?

I agree with this approach. When I think of tags, I think of these as being either system default or user created on the saved object itself. For UI specific components, I think it's fair to have different categories of groups defined by the application itself. We probably wouldn't even need to name them, just provide a quick way to filter such as the In Memory - Table Selection has here.

What about objects that are represented via plugins today, but should be migrated to be Saved Objects?

I think this is a great question and agree that user created workpad and element templates should be considered saved objects of a different type. I'm +1 to the proposal of not showing a UI initially and would be fine even starting with a default, generic, application specific category of "user created" or something along those lines.

If we go with the above proposal, that brings up the question of what will eventually be a saved object and what won't?

I think any type of Canvas plugin (function, UI, element) would be difficult here, and we'd need to consider whatever approach we take for security of plugins to be consistent across all other plugins as well (visualizations, Kibana, etc.). We did want to improve the experience of loading a plugin by simply dragging and dropping the package into the UI itself. With Canvas plugins, you do not need to restart Kibana to utilize. If this were to become a reality, I would be comfortable with limiting plugin installation to specific users (admins) and placing responsibility on the end user vs. treating them as saved objects. At least initially. Element templates to me at least, are a different story. This is an end user configuring existing elements, whether they are the default elements or plugins, and saving this configuration for reuse. To me, this configuration is worthy of a saved object that is readily available for import / export into other environments.

Question: Is the benefit of displaying tags on functions and elements worth the complexity this will possibly bring down the line? Lets also consider...

I still struggle personally to see how tags would work with functions, but that's a different story. I think we'll get pretty far with "system tags" or "categories" for now. These are essentially groupings defined by us and don't need to be thought of as Kibana wide tags.

My personal feeling is that we should punt on this feature because there isn't a lot of time till 6.7 and there are a lot of questions here. I don't think we should rush it unless we are confident we have the right path forward. Even if we had the right path forward, stability and performance, especially in the cloud environment, are top priority, more than any feature. Do we have enough time to tackle both and be confident we are implementing it correctly?

Echoing my comments throughout my feedback, I agree to an extent. Personally, I think for now it'd be fine to provide static tags, categories or groupings of elements, element templates and workpad templates in the Canvas UI. I imagine we're a bit far off from Saved Objects and see no harm in going down this route given that other features in the Kibana UI are already doing this today (examples shown above). I'd punt on customization of these for a bit while we figure out the roadmap for saved object tags. I agree that there is a certain amount of complexity and potential confusion with this feature.

Separately, I'd work towards making Canvas workpads more "official" saved objects. To start, we could add a Canvas tab to the saved object management UI. Workpad templates and element templates would be a great addition here as well as we begin to introduce these features.

Phew. Sorry for the novel. Would love your thoughts @stacey-gammon

[cqliu1]

Personally, I think for now it'd be fine to provide static tags, categories or groupings of elements, element templates and workpad templates in the Canvas UI.

I think implementing static tags/categories for elements is definitely doable for 6.7, and this would be a Canvas specific implementation. These tags would only be displayed/used to filter elements from the element selection view.

If element templates are going to be saved objects, then they won't get tagging until we can tackle Saved Object tagging.

[alexfrancoeur]

If element templates are going to be saved objects, then they won't get tagging until we can tackle Saved Object tagging.

@cqliu1 I'd be comfortable with that. Would it be possible at all to apply a default tag? Something like element template or user created that differentiates it from our defaults?

[cqliu1]

@alexfrancoeur It should be possible from a UI standpoint. If we have some way of checking if an element card to be displayed is for an element template instead of an element plugin, then we can display a template tag. We just won't store the tag on the Saved Object itself for now.

[stacey-gammon]

Echoing my comments throughout my feedback, I agree to an extent. Personally, I think for now it'd be fine to provide static tags, categories or groupings of elements, element templates and workpad templates in the Canvas UI. I imagine we're a bit far off from Saved Objects and see no harm in going down this route given that other features in the Kibana UI are already doing this today (examples shown above)

I don't think we should use the fact that other UIs are doing it as a reason for Canvas to do it. They are going to face the same issues when Saved Object Tagging comes along and face some interesting migration plans.

I imagine we're a bit far off from Saved Objects. .... Separately, I'd work towards making Canvas workpads more "official" saved objects. To start, we could add a Canvas tab to the saved object management UI.

Yes, this makes sense to me, I'd rather see this prioritized over tagging.

It just feels like to me that this feature is a very small win that will potentially create a lot of headache down the line. From my understanding the goal of tagging is to make types of elements more discoverable and filterable on. We can already filter on text and help text - how much extra do tags really get us? What if we pick a tag name that doesn't make sense to some users? Tags for organizational /discoverability purposes are best as user defined since the "right" tag name is subjective. Also, if we had these items as Saved Objects we could leverage Elasticsearch to search and highlight on text fields to improve discoverability.

[ryankeairns]

I've been following along and just wanted to +1 the direction that @stacey-gammon is pursuing.

Tagging, at this point, doesn't appear to provide much value especially given that our provided content (templates, functions, etc.) is neither deep nor growing very quickly. Adding on to that last thought, tagging is most beneficial as a UX feature when there is much for a user to sift through.

[timductive]

Closing this for merge of element tags, we can re-open if we need to.