Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Ingest Pipelines] Load from json #70297

Merged
merged 13 commits into from
Jul 3, 2020
Merged

Conversation

jloleysens
Copy link
Contributor

@jloleysens jloleysens commented Jun 30, 2020

Summary

Added basic load from JSON functionality similar to what mappings editor offers.

How to test

  1. Go to ingest pipelines app in the Kibana management section
  2. Create or edit an existing pipeline
  3. Click on the Load JSON button
  4. Paste a valid pipeline into the code editor, something like:
{
      processors: [{ set: { field: 'test', value: 123 } }],
      on_failure: [
        {
          gsub: {
            field: '_index',
            pattern: '(.monitoring-\\w+-)6(-.+)',
            replacement: '$17$2',
          },
        },
      ],
    }

And click Load and overwrite.

Notes

This functionality only does basic check to see wether we can properly deserialise the pipeline object. We still ultimately rely on ES to validate the pipeline processors for us.

Screenshots

New load json button
Screenshot 2020-06-30 at 11 11 40

load json modal
Screenshot 2020-06-30 at 11 11 47

error message about invalid pipeline object
Screenshot 2020-06-30 at 11 13 18

Invalid json error state, load button disabled
Screenshot 2020-06-30 at 11 13 35

Checklist

- refactored the pipeline processsors editor components for
  portability
- added CIT for load from json component
@jloleysens jloleysens added v8.0.0 Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more release_note:skip Skip the PR/issue when compiling release notes v7.9.0 Feature:Ingest Node Pipelines Ingest node pipelines management labels Jun 30, 2020
@jloleysens jloleysens requested a review from sebelga June 30, 2020 09:11
@jloleysens jloleysens requested a review from a team as a code owner June 30, 2020 09:11
@elasticmachine
Copy link
Contributor

Pinging @elastic/es-ui (Team:Elasticsearch UI)

@jloleysens
Copy link
Contributor Author

@elasticmachine merge upstream

@jloleysens jloleysens changed the title Load from json [Ingest Pipelines] Load from json Jun 30, 2020
Copy link
Contributor

@sebelga sebelga left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job @jloleysens ! Tested locally and works as expected. I left a few comments in the code, no real blockers.

I had a few comments on the UX, not related to the changes on this PR but leaving them here anyways 😊

  • When clicking the expand on the JSON, there is no way to go back to normal view, unless we know we can hit ESCAPE. I am not sure I see the value to be able to go fullscreen.

Screenshot 2020-07-01 at 13 25 08

  • Flyout to edit: I personally think it's strange to have the JSON editor in white background and prefer the gray background that we have everywhere. Also, I would prefer to use the FlyoutFooter bar for the actions ("Cancel" + "Update"), like the mappings editor.

Screenshot 2020-07-01 at 13 27 15

Screenshot 2020-07-01 at 14 28 38

  • Do you think we could reduce the height of the JSON editor so we can see the "Invalid JSON" error below it? As you can see here, the JSON is invalid, but I don't see it and the button is grayed out, and unless you scroll you don't know why.

Screenshot 2020-07-01 at 13 37 43

  • I think it would be better to limit the width of the flyout as we do in the mappings editor.

Screenshot 2020-07-01 at 13 43 08

Screenshot 2020-07-01 at 13 42 46

@jloleysens
Copy link
Contributor Author

@elasticmachine merge upstream

@jloleysens
Copy link
Contributor Author

@elasticmachine merge upstream

@jloleysens
Copy link
Contributor Author

@elasticmachine merge upstream

@jloleysens
Copy link
Contributor Author

Hm, the fleet functional test keeps failing. That or another, unrelated functional test (lens). Retrying...

@jloleysens
Copy link
Contributor Author

@elasticmachine merge upstream

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Build metrics

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@jloleysens jloleysens merged commit 5159635 into elastic:master Jul 3, 2020
@jloleysens jloleysens deleted the load_from_json branch July 3, 2020 08:43
gmmorris added a commit to gmmorris/kibana that referenced this pull request Jul 3, 2020
* master: (32 commits)
  [Ingest Pipelines] Load from json (elastic#70297)
  [Rum Dashbaord] Rum selected service view (elastic#70579)
  [Uptime] Prevent duplicate requests on load for index status (elastic#70585)
  [ML] Changing shared module setup function parameters (elastic#70589)
  [Ingest Manager] Add ability to sort to agent configs and package configs (elastic#70676)
  [Alerting] document requirements for developing new action types (elastic#69164)
  Fixed adding an extra space character on selecting alert variable in action text fields (elastic#70028)
  [Maps] show vector tile labels on top (elastic#69444)
  chore(NA): upgrade to lodash@4 (elastic#69868)
  Add Snapshot Restore README with quick-testing steps. (elastic#70494)
  [EPM] Use higher priority than default templates (elastic#70640)
  [Maps] Fix cannot select Solid fill-color when removing fields (elastic#70621)
  [kbn/optimizer] only build specified themes (elastic#70389)
  Fix saved query modal overlay (elastic#68826)
  Update component templates list to render empty prompt inside of content container. Show detail panel when deep-linked, even if there are no component templates. (elastic#70633)
  [Security Solution] Renames the `Investigate in Resolver` Timeline action (elastic#70634)
  fix 400 error on initial signals search (elastic#70618)
  [Maps] fix unable to edit heatmap metric (elastic#70606)
  Update network idle timeout (elastic#70629)
  [APM] Disable flaky useFetcher test (elastic#70638)
  ...
gmmorris added a commit to gmmorris/kibana that referenced this pull request Jul 3, 2020
* master: (199 commits)
  [Telemetry] Add documentation about Application Usage (elastic#70624)
  [Ingest Manager] Improve agent unenrollment with unenroll action (elastic#70031)
  Handle timeouts on creating templates (elastic#70635)
  [Lens] Add ability to set colors for y-axis series (elastic#70311)
  [Uptime] Use elastic charts donut (elastic#70364)
  [Ingest Manager] Update registry URL to point to snapshot registry (elastic#70687)
  [Composable template] Create / Edit wizard (elastic#70220)
  [APM] Optimize services overview (elastic#69648)
  [Ingest Pipelines] Load from json (elastic#70297)
  [Rum Dashbaord] Rum selected service view (elastic#70579)
  [Uptime] Prevent duplicate requests on load for index status (elastic#70585)
  [ML] Changing shared module setup function parameters (elastic#70589)
  [Ingest Manager] Add ability to sort to agent configs and package configs (elastic#70676)
  [Alerting] document requirements for developing new action types (elastic#69164)
  Fixed adding an extra space character on selecting alert variable in action text fields (elastic#70028)
  [Maps] show vector tile labels on top (elastic#69444)
  chore(NA): upgrade to lodash@4 (elastic#69868)
  Add Snapshot Restore README with quick-testing steps. (elastic#70494)
  [EPM] Use higher priority than default templates (elastic#70640)
  [Maps] Fix cannot select Solid fill-color when removing fields (elastic#70621)
  ...
jloleysens added a commit to jloleysens/kibana that referenced this pull request Jul 4, 2020
* WiP load from json modal ready, need to refactor more stuff

* First iteration of load from JSON functionality

- refactored the pipeline processsors editor components for
  portability
- added CIT for load from json component

* added comment

* update deserialize with tests and make it more fault tolerant

* use flyout footer

* remove console.error and make the json editor a lot shorter

* address PR feedback

- Update form schema and form schema types
- simplify the save handler
- refactor processors_title to processors_header

* remove unused translations

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
jloleysens added a commit that referenced this pull request Jul 4, 2020
* WiP load from json modal ready, need to refactor more stuff

* First iteration of load from JSON functionality

- refactored the pipeline processsors editor components for
  portability
- added CIT for load from json component

* added comment

* update deserialize with tests and make it more fault tolerant

* use flyout footer

* remove console.error and make the json editor a lot shorter

* address PR feedback

- Update form schema and form schema types
- simplify the save handler
- refactor processors_title to processors_header

* remove unused translations

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Ingest Node Pipelines Ingest node pipelines management release_note:skip Skip the PR/issue when compiling release notes Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more v7.9.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants