Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dynamically generate ingest node pipeline processor descriptions #95486

Closed
cjcenizal opened this issue Mar 25, 2021 · 4 comments · Fixed by #95507
Closed

Dynamically generate ingest node pipeline processor descriptions #95486

cjcenizal opened this issue Mar 25, 2021 · 4 comments · Fixed by #95507
Labels
enhancement New value added to drive a business result Feature:Ingest Node Pipelines Ingest node pipelines management Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more

Comments

@cjcenizal
Copy link
Contributor

cjcenizal commented Mar 25, 2021

In #94432, a user mentioned how useful processor descriptions would be for the pipelines shipped by Elastic. I've opened elastic/integrations#807 to track the work for doing that work. The target UX we're aiming for is to provide a higher-level description of what each processors is intended to achieve at a glance.

Screenshot 2021-03-16 at 10 01 34

However, @andrewkroh (comment) and @P1llus (comment) each made a similar suggestions that many processors are so simple that we should dynamically generate descriptions of what they're doing. This would save time for package maintainers because they'll only need to add descriptions for the more complex processors, such as grok, script, and dissect. It will also benefit users who are creating their own pipelines because they'll similarly be able to save time.

Examples

The rename processor: "Rename {field} to {target_field}"
The convert processor: "Convert {field} to {type}"

Requirements

  1. Processors with empty or missing descriptions should be described by dynamic descriptions.
  2. Dynamic descriptions should be presented both for each processor in the tree, and also for the processor that's open in the Editor Processor flyout.
@cjcenizal cjcenizal added enhancement New value added to drive a business result Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more Feature:Ingest Node Pipelines Ingest node pipelines management labels Mar 25, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/es-ui (Team:Elasticsearch UI)

@cjcenizal
Copy link
Contributor Author

CC @jamiehynds

@ycombinator
Copy link
Contributor

ycombinator commented Mar 26, 2021

One consideration might be to have Elasticsearch provide the default descriptions, maybe as part of the GET Pipeline API. This way not just the Ingest Node Editor UI but any other consumers of Ingest pipelines (e.g. Dev Tools > Console, potentially elastic-package) could also benefit and the user would see the same descriptions no matter which tool was used to interact with Ingest pipelines.

@cjcenizal
Copy link
Contributor Author

cjcenizal commented Mar 26, 2021

Thanks for suggesting this @ycombinator! I had a similar thought when I was creating this issue but ended up backing away from it. My primary concern was that users won't be able to discern between user-authored descriptions and dynamically generated descriptions, potentially confusing people. It seems unexpected for the API to independently define content that is intended for authors to customize.

Some secondary concerns: What if the authors speak a language other than English? ES doesn't localize. How frustrating that would be, to be bombarded with noise that's not in your language, with no way to disable it -- potentially even a hostile UX. What if authors want the description to actually be empty? When reading the JSON, the user has access to all of the details of a processor's behavior, so for "obvious" processors some users might see the descriptions as noise.

In the end, it seemed to me like a more targeted solution in Kibana will allow us to safely bypass these concerns and solve the original problem more quickly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New value added to drive a business result Feature:Ingest Node Pipelines Ingest node pipelines management Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants