Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[7.x] [data views] Move data views api from data plugin and into its own (#113497) #113833

Merged
merged 2 commits into from
Oct 4, 2021
Merged

[7.x] [data views] Move data views api from data plugin and into its own (#113497) #113833

merged 2 commits into from
Oct 4, 2021

Conversation

mattkime
Copy link
Contributor

@mattkime mattkime commented Oct 4, 2021

Backports the following commits to 7.x:

@mattkime @kibanamachine
[data views] Move data views api from data plugin and into its own (e…
629252c 
…lastic#113497)

* initial pass at moving data views into own plugin

* require expressions plugin, fix META_FIELDS reference

* bundle limits and localization

* fix integration test

* update plugin list and jest config

* type fixes

* search fixes

* fix localization

* fix mocks

* fix mocks

* fix stub

* type fixes

* fix import on test file

* path fixes

* remove shorted dotted from data plugin

* more todo removal

* eslint fixes

* eslint fix

* simplify data views server plugin

* simplify data views server plugin

* simplify data views server plugin

* fix imports on api routes

* fix imports on api routes

* update plugin list

* ts fixes

* ts fixes

* add deprecation notice

* fix circular dependency and api integration test

* fix circular dependency and api integration test

* rename types for better clarity

* path fixes

* jest.config and tsconfig cleanup

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
@mattkime mattkime enabled auto-merge (squash) October 4, 2021 18:48
@kibanamachine
Copy link
Contributor

💛 Build succeeded, but was flaky

Test Failures

Kibana Pipeline / general / Performance Tests.x-pack/test/performance/tests/reporting_dashboard·ts.performance reporting dashbaord downloaded PDF has OK status

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has failed 3 times on tracked branches: https://github.com/elastic/kibana/issues/110470

[00:00:00]                         │
[00:00:00]                           └-: performance
[00:00:00]                             └-> "before all" hook in "performance"
[00:00:00]                             └-: reporting dashbaord
[00:00:00]                               └-> "before all" hook for "downloaded PDF has OK status"
[00:00:00]                               └-> "before all" hook for "downloaded PDF has OK status"
[00:00:00]                                 │ debg resolved import for x-pack/test/performance/kbn_archives/reporting_dashboard to /dev/shm/workspace/parallel/8/kibana/x-pack/test/performance/kbn_archives/reporting_dashboard.json
[00:00:00]                                 │ info importing 4 saved objects { space: undefined }
[00:00:00]                                 │ info [o.e.c.m.MetadataMappingService] [node-01] [.kibana_7.16.0_001/s_bA0vs4RjehhJFyW6x2Nw] update_mapping [_doc]
[00:00:00]                                 │ info [o.e.c.m.MetadataMappingService] [node-01] [.kibana_7.16.0_001/s_bA0vs4RjehhJFyW6x2Nw] update_mapping [_doc]
[00:00:00]                                 │ info [o.e.c.m.MetadataMappingService] [node-01] [.kibana_7.16.0_001/s_bA0vs4RjehhJFyW6x2Nw] update_mapping [_doc]
[00:00:00]                                 │ info [o.e.c.m.MetadataMappingService] [node-01] [.kibana_7.16.0_001/s_bA0vs4RjehhJFyW6x2Nw] update_mapping [_doc]
[00:00:00]                                 │ info [o.e.c.m.MetadataMappingService] [node-01] [.kibana_7.16.0_001/s_bA0vs4RjehhJFyW6x2Nw] update_mapping [_doc]
[00:00:00]                                 │ succ import success
[00:00:00]                                 │ info [x-pack/test/performance/es_archives/reporting_dashboard] Loading "mappings.json"
[00:00:00]                                 │ info [x-pack/test/performance/es_archives/reporting_dashboard] Loading "data.json.gz"
[00:00:00]                                 │ info [o.e.c.m.MetadataCreateIndexService] [node-01] [foo] creating index, cause [api], templates [], shards [1]/[1]
[00:00:00]                                 │ info [x-pack/test/performance/es_archives/reporting_dashboard] Created index "foo"
[00:00:00]                                 │ debg [x-pack/test/performance/es_archives/reporting_dashboard] "foo" settings {"index":{"number_of_replicas":"1","number_of_shards":"1"}}
[00:00:02]                                 │ info [x-pack/test/performance/es_archives/reporting_dashboard] Indexed 10000 docs into "foo"
[00:00:02]                               └-> downloaded PDF has OK status
[00:00:02]                                 └-> "before each" hook: global before each for "downloaded PDF has OK status"
[00:00:02]                                 │ debg navigating to dashboards url: http://localhost:6181/app/dashboards
[00:00:02]                                 │ debg navigate to: http://localhost:6181/app/dashboards
[00:00:02]                                 │ debg browser[INFO] http://localhost:6181/login?next=%2Fapp%2Fdashboards%3F_t%3D1633380580829 281 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:00:02]                                 │
[00:00:02]                                 │ debg browser[INFO] http://localhost:6181/bootstrap.js 41:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:00:02]                                 │ debg ... sleep(700) start
[00:00:03]                                 │ debg ... sleep(700) end
[00:00:03]                                 │ debg returned from get, calling refresh
[00:00:04]                                 │ debg browser[INFO] http://localhost:6181/login?next=%2Fapp%2Fdashboards%3F_t%3D1633380580829 281 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:00:04]                                 │
[00:00:04]                                 │ debg browser[INFO] http://localhost:6181/bootstrap.js 41:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:00:05]                                 │ debg currentUrl = http://localhost:6181/login?next=%2Fapp%2Fdashboards%3F_t%3D1633380580829
[00:00:05]                                 │          appUrl = http://localhost:6181/app/dashboards
[00:00:05]                                 │ debg TestSubjects.find(kibanaChrome)
[00:00:05]                                 │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"]') with timeout=60000
[00:00:05]                                 │ debg Found login page
[00:00:05]                                 │ debg TestSubjects.setValue(loginUsername, test_user)
[00:00:05]                                 │ debg TestSubjects.click(loginUsername)
[00:00:05]                                 │ debg Find.clickByCssSelector('[data-test-subj="loginUsername"]') with timeout=10000
[00:00:05]                                 │ debg Find.findByCssSelector('[data-test-subj="loginUsername"]') with timeout=10000
[00:00:06]                                 │ERROR browser[SEVERE] http://localhost:6181/api/licensing/info - Failed to load resource: the server responded with a status of 401 (Unauthorized)
[00:00:06]                                 │ debg TestSubjects.setValue(loginPassword, changeme)
[00:00:06]                                 │ debg TestSubjects.click(loginPassword)
[00:00:06]                                 │ debg Find.clickByCssSelector('[data-test-subj="loginPassword"]') with timeout=10000
[00:00:06]                                 │ debg Find.findByCssSelector('[data-test-subj="loginPassword"]') with timeout=10000
[00:00:06]                                 │ debg TestSubjects.click(loginSubmit)
[00:00:06]                                 │ debg Find.clickByCssSelector('[data-test-subj="loginSubmit"]') with timeout=10000
[00:00:06]                                 │ debg Find.findByCssSelector('[data-test-subj="loginSubmit"]') with timeout=10000
[00:00:06]                                 │ debg Find.waitForDeletedByCssSelector('.kibanaWelcomeLogo') with timeout=10000
[00:00:06]                                 │ proc [kibana]   log   [20:49:44.458] [info][plugins][routes][security] Logging in with provider "basic" (basic)
[00:00:06]                                 │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"]') with timeout=60000
[00:00:06]                                 │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"] nav:not(.ng-hide)') with timeout=60000
[00:00:09]                                 │ debg browser[INFO] http://localhost:6181/app/dashboards?_t=1633380580829 281 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:00:09]                                 │
[00:00:09]                                 │ debg browser[INFO] http://localhost:6181/bootstrap.js 41:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:00:10]                                 │ debg browser[INFO] http://localhost:6181/app/dashboards?_t=1633380587737 281 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:00:10]                                 │
[00:00:10]                                 │ debg browser[INFO] http://localhost:6181/bootstrap.js 41:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:00:11]                                 │ debg Finished login process currentUrl = http://localhost:6181/app/dashboards
[00:00:11]                                 │ debg ... sleep(501) start
[00:00:11]                                 │ debg ... sleep(501) end
[00:00:11]                                 │ debg in navigateTo url = http://localhost:6181/app/dashboards
[00:00:11]                                 │ debg Waiting up to 20000ms for dashboard landing page...
[00:00:11]                                 │ debg onDashboardLandingPage
[00:00:11]                                 │ debg TestSubjects.exists(dashboardLandingPage)
[00:00:11]                                 │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="dashboardLandingPage"]') with timeout=5000
[00:00:12]                                 │ debg Load Saved Dashboard dashboard
[00:00:12]                                 │ debg gotoDashboardLandingPage
[00:00:12]                                 │ debg onDashboardLandingPage
[00:00:12]                                 │ debg TestSubjects.exists(dashboardLandingPage)
[00:00:12]                                 │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="dashboardLandingPage"]') with timeout=5000
[00:00:12]                                 │ debg searchForItemWithName: dashboard
[00:00:12]                                 │ debg TestSubjects.find(tableListSearchBox)
[00:00:12]                                 │ debg Find.findByCssSelector('[data-test-subj="tableListSearchBox"]') with timeout=10000
[00:00:12]                                 │ debg isGlobalLoadingIndicatorVisible
[00:00:12]                                 │ debg TestSubjects.exists(globalLoadingIndicator)
[00:00:12]                                 │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="globalLoadingIndicator"]') with timeout=1500
[00:00:12]                                 │ debg TestSubjects.exists(globalLoadingIndicator-hidden)
[00:00:12]                                 │ debg Find.existsByCssSelector('[data-test-subj="globalLoadingIndicator-hidden"]') with timeout=100000
[00:00:15]                                 │ debg TestSubjects.click(dashboardListingTitleLink-dashboard)
[00:00:15]                                 │ debg Find.clickByCssSelector('[data-test-subj="dashboardListingTitleLink-dashboard"]') with timeout=10000
[00:00:15]                                 │ debg Find.findByCssSelector('[data-test-subj="dashboardListingTitleLink-dashboard"]') with timeout=10000
[00:00:15]                                 │ debg isGlobalLoadingIndicatorVisible
[00:00:15]                                 │ debg TestSubjects.exists(globalLoadingIndicator)
[00:00:15]                                 │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="globalLoadingIndicator"]') with timeout=1500
[00:00:15]                                 │ debg browser[INFO] http://localhost:6181/app/dashboards#/view/37b49c50-2dc6-11eb-8af3-cb3aa84dbabd?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-15m,to:now)) 281 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:00:15]                                 │
[00:00:15]                                 │ debg browser[INFO] http://localhost:6181/bootstrap.js 41:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:00:16]                                 │ debg --- retry.tryForTime error: [data-test-subj="globalLoadingIndicator"] is not displayed
[00:00:17]                                 │ debg TestSubjects.exists(globalLoadingIndicator-hidden)
[00:00:17]                                 │ debg Find.existsByCssSelector('[data-test-subj="globalLoadingIndicator-hidden"]') with timeout=100000
[00:00:17]                                 │ debg TestSubjects.missingOrFail(dashboardLandingPage)
[00:00:17]                                 │ debg Find.waitForDeletedByCssSelector('[data-test-subj="dashboardLandingPage"]') with timeout=10000
[00:00:17]                                 │ debg openPdfReportingPanel
[00:00:17]                                 │ debg openShareMenuItem title:PDF Reports
[00:00:17]                                 │ debg TestSubjects.exists(shareContextMenu)
[00:00:17]                                 │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="shareContextMenu"]') with timeout=2500
[00:00:18]                                 │ info [o.e.c.m.MetadataCreateIndexService] [node-01] [.async-search] creating index, cause [api], templates [], shards [1]/[0]
[00:00:20]                                 │ debg --- retry.tryForTime error: [data-test-subj="shareContextMenu"] is not displayed
[00:00:21]                                 │ info [o.e.c.m.MetadataMappingService] [node-01] [.kibana_7.16.0_001/s_bA0vs4RjehhJFyW6x2Nw] update_mapping [_doc]
[00:00:21]                                 │ debg TestSubjects.click(shareTopNavButton)
[00:00:21]                                 │ debg Find.clickByCssSelector('[data-test-subj="shareTopNavButton"]') with timeout=10000
[00:00:21]                                 │ debg Find.findByCssSelector('[data-test-subj="shareTopNavButton"]') with timeout=10000
[00:00:22]                                 │ debg Find.findByCssSelector('div.euiContextMenuPanel') with timeout=10000
[00:00:22]                                 │ debg TestSubjects.click(sharePanel-PDFReports)
[00:00:22]                                 │ debg Find.clickByCssSelector('[data-test-subj="sharePanel-PDFReports"]') with timeout=10000
[00:00:22]                                 │ debg Find.findByCssSelector('[data-test-subj="sharePanel-PDFReports"]') with timeout=10000
[00:00:23]                                 │ debg Find.waitForElementStale with timeout=10000
[00:00:23]                                 │ debg TestSubjects.click(generateReportButton)
[00:00:23]                                 │ debg Find.clickByCssSelector('[data-test-subj="generateReportButton"]') with timeout=10000
[00:00:23]                                 │ debg Find.findByCssSelector('[data-test-subj="generateReportButton"]') with timeout=10000
[00:00:24]                                 │ info [o.e.c.m.MetadataMappingService] [node-01] [.kibana_7.16.0_001/s_bA0vs4RjehhJFyW6x2Nw] update_mapping [_doc]
[00:00:24]                                 │ debg getReportURL
[00:00:24]                                 │ debg TestSubjects.getAttribute(downloadCompletedReportButton, href, tryTimeout=120000, findTimeout=60000)
[00:00:24]                                 │ debg TestSubjects.find(downloadCompletedReportButton)
[00:00:24]                                 │ debg Find.findByCssSelector('[data-test-subj="downloadCompletedReportButton"]') with timeout=60000
[00:01:25]                                 │ debg --- retry.tryForTime error: Waiting for element to be located By(css selector, [data-test-subj="downloadCompletedReportButton"])
[00:01:25]                                 │      Wait timed out after 61179ms
[00:01:25]                                 │ debg TestSubjects.find(downloadCompletedReportButton)
[00:01:25]                                 │ debg Find.findByCssSelector('[data-test-subj="downloadCompletedReportButton"]') with timeout=60000
[00:02:27]                                 │ debg --- retry.tryForTime error: Waiting for element to be located By(css selector, [data-test-subj="downloadCompletedReportButton"])
[00:02:27]                                 │      Wait timed out after 61201ms
[00:02:27]                                 │ info Taking screenshot "/dev/shm/workspace/parallel/8/kibana/x-pack/test/functional/screenshots/failure/performance reporting dashbaord downloaded PDF has OK status.png"
[00:02:27]                                 │ info Current URL is: http://localhost:6181/app/dashboards#/view/37b49c50-2dc6-11eb-8af3-cb3aa84dbabd?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-15m,to:now))
[00:02:27]                                 │ info Saving page source to: /dev/shm/workspace/parallel/8/kibana/x-pack/test/performance/failure_debug/html/performance reporting dashbaord downloaded PDF has OK status.html
[00:02:27]                                 └- ✖ fail: performance reporting dashbaord downloaded PDF has OK status
[00:02:27]                                 │      Error: retry.tryForTime timeout: TimeoutError: Waiting for element to be located By(css selector, [data-test-subj="downloadCompletedReportButton"])
[00:02:27]                                 │ Wait timed out after 61201ms
[00:02:27]                                 │     at /dev/shm/workspace/parallel/8/kibana/node_modules/selenium-webdriver/lib/webdriver.js:842:17
[00:02:27]                                 │     at runMicrotasks (<anonymous>)
[00:02:27]                                 │     at processTicksAndRejections (internal/process/task_queues.js:95:5)
[00:02:27]                                 │       at onFailure (/dev/shm/workspace/parallel/8/kibana/test/common/services/retry/retry_for_success.ts:17:9)
[00:02:27]                                 │       at retryForSuccess (/dev/shm/workspace/parallel/8/kibana/test/common/services/retry/retry_for_success.ts:57:13)
[00:02:27]                                 │       at RetryService.tryForTime (/dev/shm/workspace/parallel/8/kibana/test/common/services/retry/retry.ts:22:12)
[00:02:27]                                 │       at TestSubjects.getAttribute (/dev/shm/workspace/parallel/8/kibana/test/functional/services/common/test_subjects.ts:190:12)
[00:02:27]                                 │       at ReportingPageObject.getReportURL (test/functional/page_objects/reporting_page.ts:36:17)
[00:02:27]                                 │       at Context.<anonymous> (test/performance/tests/reporting_dashboard.ts:50:7)
[00:02:27]                                 │       at Object.apply (/dev/shm/workspace/parallel/8/kibana/node_modules/@kbn/test/target_node/functional_test_runner/lib/mocha/wrap_function.js:87:16)
[00:02:27]                                 │ 
[00:02:27]                                 │

Stack Trace

Error: retry.tryForTime timeout: TimeoutError: Waiting for element to be located By(css selector, [data-test-subj="downloadCompletedReportButton"])
Wait timed out after 61201ms
    at /dev/shm/workspace/parallel/8/kibana/node_modules/selenium-webdriver/lib/webdriver.js:842:17
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
    at onFailure (/dev/shm/workspace/parallel/8/kibana/test/common/services/retry/retry_for_success.ts:17:9)
    at retryForSuccess (/dev/shm/workspace/parallel/8/kibana/test/common/services/retry/retry_for_success.ts:57:13)
    at RetryService.tryForTime (/dev/shm/workspace/parallel/8/kibana/test/common/services/retry/retry.ts:22:12)
    at TestSubjects.getAttribute (/dev/shm/workspace/parallel/8/kibana/test/functional/services/common/test_subjects.ts:190:12)
    at ReportingPageObject.getReportURL (test/functional/page_objects/reporting_page.ts:36:17)
    at Context.<anonymous> (test/performance/tests/reporting_dashboard.ts:50:7)
    at Object.apply (/dev/shm/workspace/parallel/8/kibana/node_modules/@kbn/test/target_node/functional_test_runner/lib/mocha/wrap_function.js:87:16)

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
data 573 537 -36
dataViews - 44 +44
total +8

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
data 2815 2809 -6
dataViews - 545 +545
total +539

Any counts in public APIs

Total count of every any typed public API. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats any for more detailed information.

id before after diff
data 44 43 -1
dataViews - 6 +6
total +5

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id before after diff
data 49 48 -1
dataViews - 5 +5
total +4

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
data 500.4KB 472.2KB -28.1KB
dataViews - 40.5KB +40.5KB
visualizations 33.4KB 33.4KB +20.0B
total +12.3KB
Unknown metric groups

API count

id before after diff
data 3201 3194 -7
dataViews - 685 +685
total +678

References to deprecated APIs

id before after diff
apm 57 73 +16
canvas 64 67 +3
dashboard 217 240 +23
data 2 463 +461
dataViews - 235 +235
dataVisualizer 146 242 +96
discover 1135 1790 +655
fleet 33 39 +6
graph 102 168 +66
indexPatternEditor 53 85 +32
indexPatternFieldEditor 100 156 +56
indexPatternManagement 306 496 +190
infra 358 507 +149
inputControlVis 209 325 +116
lens 246 346 +100
maps 1146 1659 +513
ml 466 704 +238
monitoring 39 53 +14
observability 112 166 +54
osquery 22 34 +12
reporting 25 29 +4
savedObjects 26 44 +18
savedObjectsManagement 74 110 +36
security 61 71 +10
securitySolution 878 1008 +130
stackAlerts 130 183 +53
timelines 75 84 +9
transform 96 148 +52
uptime 16 26 +10
visDefaultEditor 141 223 +82
visTypeTimelion 36 45 +9
visTypeTimeseries 226 347 +121
visTypeVega 29 35 +6
visualizations 89 104 +15
visualize 93 99 +6
total +3596

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@mattkime mattkime merged commit 0d2fcbc into elastic:7.x Oct 4, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
backport
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@mattkime @kibanamachine