[Dashboard] Keep managed panels out of unmanaged dashboards

[drewdaemon]

Summary

Close #172383
Close #172384

This PR introduces a new embeddable-related registry which clients can use to define a custom transformation from saved object to embeddable input during the add-panel-from-library sequence.

Then, each content type uses this to communicate whether a particular object should be added by-ref or by-val based on the presence of managed: true on the saved object (example).

Managed panels are added by-value to dashboards

Cloning a managed dashboard inlines all by-ref panels

Screen.Recording.2024-02-01.at.3.07.47.PM.mov

Testing material

Here's a dashboard with a bunch of managed by-ref panels.

{"attributes":{"allowHidden":false,"fieldAttrs":"{}","fieldFormatMap":"{}","fields":"[]","name":"logstash-*","runtimeFieldMap":"{}","sourceFilters":"[]","timeFieldName":"@timestamp","title":"logstash-*"},"coreMigrationVersion":"8.8.0","created_at":"2024-02-01T22:06:58.824Z","id":"5f863f70-4728-4e8d-b441-db08f8c33b28","managed":false,"references":[],"type":"index-pattern","typeMigrationVersion":"8.0.0","updated_at":"2024-02-01T22:06:58.824Z","version":"WzQxNCwxXQ=="}
{"attributes":{"description":"","kibanaSavedObjectMeta":{"searchSourceJSON":"{\"query\":{\"query\":\"\",\"language\":\"kuery\"},\"filter\":[]}"},"title":"Managed legacy visualization","uiStateJSON":"{}","version":1,"visState":"{\"title\":\"Legacy visualization\",\"type\":\"metrics\",\"aggs\":[],\"params\":{\"id\":\"1a14d0ad-0d74-4470-a189-8f040cddc1a1\",\"type\":\"timeseries\",\"series\":[{\"id\":\"daa8bbf7-86cc-4394-b249-be48da9f7351\",\"color\":\"#68BC00\",\"split_mode\":\"everything\",\"palette\":{\"type\":\"palette\",\"name\":\"default\"},\"metrics\":[{\"id\":\"795375d9-1aa6-454d-9b23-687e69f3382c\",\"type\":\"count\"}],\"separate_axis\":0,\"axis_position\":\"right\",\"formatter\":\"default\",\"chart_type\":\"line\",\"line_width\":1,\"point_size\":1,\"fill\":0.5,\"stacked\":\"none\",\"override_index_pattern\":0,\"series_drop_last_bucket\":0}],\"time_field\":\"\",\"use_kibana_indexes\":true,\"interval\":\"\",\"axis_position\":\"left\",\"axis_formatter\":\"number\",\"axis_scale\":\"normal\",\"show_legend\":1,\"truncate_legend\":1,\"max_lines_legend\":1,\"show_grid\":1,\"tooltip_mode\":\"show_all\",\"drop_last_bucket\":0,\"index_pattern_ref_name\":\"metrics_0_index_pattern\"}}"},"coreMigrationVersion":"8.8.0","created_at":"2024-02-01T22:06:58.824Z","id":"managed-feb9-4ba6-9538-1b8f67fb4f57","managed":true,"references":[{"id":"5f863f70-4728-4e8d-b441-db08f8c33b28","name":"metrics_0_index_pattern","type":"index-pattern"}],"type":"visualization","typeMigrationVersion":"8.5.0","updated_at":"2024-02-01T22:06:58.824Z","version":"WzQxOSwxXQ=="}
{"attributes":{"description":"","state":{"adHocDataViews":{},"datasourceStates":{"formBased":{"layers":{"e633b1af-3ab4-4bf5-8faa-fefde06c4a4a":{"columnOrder":["f2555a1a-6f93-43fd-bc63-acdfadd47729","d229daf9-9658-4579-99af-01d8adb2f25f"],"columns":{"d229daf9-9658-4579-99af-01d8adb2f25f":{"dataType":"number","isBucketed":false,"label":"Median of bytes","operationType":"median","params":{"emptyAsNull":true},"scale":"ratio","sourceField":"bytes"},"f2555a1a-6f93-43fd-bc63-acdfadd47729":{"dataType":"date","isBucketed":true,"label":"@timestamp","operationType":"date_histogram","params":{"dropPartials":false,"includeEmptyRows":true,"interval":"auto"},"scale":"interval","sourceField":"@timestamp"}},"incompleteColumns":{},"sampling":1}}},"indexpattern":{"layers":{}},"textBased":{"layers":{}}},"filters":[],"internalReferences":[],"query":{"language":"kuery","query":""},"visualization":{"axisTitlesVisibilitySettings":{"x":true,"yLeft":true,"yRight":true},"fittingFunction":"None","gridlinesVisibilitySettings":{"x":true,"yLeft":true,"yRight":true},"labelsOrientation":{"x":0,"yLeft":0,"yRight":0},"layers":[{"accessors":["d229daf9-9658-4579-99af-01d8adb2f25f"],"layerId":"e633b1af-3ab4-4bf5-8faa-fefde06c4a4a","layerType":"data","position":"top","seriesType":"bar_stacked","showGridlines":false,"xAccessor":"f2555a1a-6f93-43fd-bc63-acdfadd47729"}],"legend":{"isVisible":true,"position":"right"},"preferredSeriesType":"bar_stacked","tickLabelsVisibilitySettings":{"x":true,"yLeft":true,"yRight":true},"valueLabels":"hide"}},"title":"Managed lens vis","visualizationType":"lnsXY"},"coreMigrationVersion":"8.8.0","created_at":"2024-02-01T22:06:58.824Z","id":"managed-36db-4a3b-a4ba-7a64ab8f130b","managed":true,"references":[{"id":"5f863f70-4728-4e8d-b441-db08f8c33b28","name":"indexpattern-datasource-layer-e633b1af-3ab4-4bf5-8faa-fefde06c4a4a","type":"index-pattern"}],"type":"lens","typeMigrationVersion":"8.9.0","updated_at":"2024-02-01T22:06:58.824Z","version":"WzQxNSwxXQ=="}
{"attributes":{"description":"","layerListJSON":"[{\"locale\":\"autoselect\",\"sourceDescriptor\":{\"type\":\"EMS_TMS\",\"isAutoSelect\":true,\"lightModeDefault\":\"road_map_desaturated\"},\"id\":\"5ff9c98e-e0d3-4aff-ac98-b33c191496b4\",\"label\":null,\"minZoom\":0,\"maxZoom\":24,\"alpha\":1,\"visible\":true,\"style\":{\"type\":\"EMS_VECTOR_TILE\",\"color\":\"\"},\"includeInFitToBounds\":true,\"type\":\"EMS_VECTOR_TILE\"}]","mapStateJSON":"{\"adHocDataViews\":[],\"zoom\":1.4,\"center\":{\"lon\":0,\"lat\":19.94277},\"timeFilters\":{\"from\":\"now-15m\",\"to\":\"now\"},\"refreshConfig\":{\"isPaused\":true,\"interval\":60000},\"query\":{\"query\":\"\",\"language\":\"kuery\"},\"filters\":[],\"settings\":{\"autoFitToDataBounds\":false,\"backgroundColor\":\"#ffffff\",\"customIcons\":[],\"disableInteractive\":false,\"disableTooltipControl\":false,\"hideToolbarOverlay\":false,\"hideLayerControl\":false,\"hideViewControl\":false,\"initialLocation\":\"LAST_SAVED_LOCATION\",\"fixedLocation\":{\"lat\":0,\"lon\":0,\"zoom\":2},\"browserLocation\":{\"zoom\":2},\"keydownScrollZoom\":false,\"maxZoom\":24,\"minZoom\":0,\"showScaleControl\":false,\"showSpatialFilters\":true,\"showTimesliderToggleButton\":true,\"spatialFiltersAlpa\":0.3,\"spatialFiltersFillColor\":\"#DA8B45\",\"spatialFiltersLineColor\":\"#DA8B45\"}}","title":"Managed map","uiStateJSON":"{\"isLayerTOCOpen\":true,\"openTOCDetails\":[]}"},"coreMigrationVersion":"8.8.0","created_at":"2024-02-01T22:06:58.824Z","id":"managed-d7ab-46eb-a807-8fed28ed8566","managed":true,"references":[],"type":"map","typeMigrationVersion":"8.4.0","updated_at":"2024-02-01T22:06:58.824Z","version":"WzQyMSwxXQ=="}
{"attributes":{"columns":["@tags","clientip"],"description":"","grid":{},"hideChart":false,"isTextBasedQuery":false,"kibanaSavedObjectMeta":{"searchSourceJSON":"{\"query\":{\"query\":\"agent.raw:\\\"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)\\\" \",\"language\":\"kuery\"},\"filter\":[],\"indexRefName\":\"kibanaSavedObjectMeta.searchSourceJSON.index\"}"},"sort":[["@timestamp","desc"]],"timeRestore":false,"title":"Managed saved search","usesAdHocDataView":false},"coreMigrationVersion":"8.8.0","created_at":"2024-02-01T22:06:58.824Z","id":"managed-3d62-4113-ac7c-de2e20a68fbc","managed":true,"references":[{"id":"5f863f70-4728-4e8d-b441-db08f8c33b28","name":"kibanaSavedObjectMeta.searchSourceJSON.index","type":"index-pattern"}],"type":"search","typeMigrationVersion":"8.0.0","updated_at":"2024-02-01T22:06:58.824Z","version":"WzQxNywxXQ=="}
{"attributes":{"description":"","kibanaSavedObjectMeta":{"searchSourceJSON":"{\"query\":{\"query\":\"\",\"language\":\"kuery\"},\"filter\":[]}"},"optionsJSON":"{\"useMargins\":true,\"syncColors\":false,\"syncCursor\":true,\"syncTooltips\":false,\"hidePanelTitles\":false}","panelsJSON":"[{\"type\":\"visualization\",\"gridData\":{\"x\":0,\"y\":0,\"w\":24,\"h\":15,\"i\":\"536c52e4-ecf1-4cde-9323-cc1c3de1fdd2\"},\"panelIndex\":\"536c52e4-ecf1-4cde-9323-cc1c3de1fdd2\",\"embeddableConfig\":{\"enhancements\":{}},\"panelRefName\":\"panel_536c52e4-ecf1-4cde-9323-cc1c3de1fdd2\"},{\"type\":\"lens\",\"gridData\":{\"x\":24,\"y\":0,\"w\":24,\"h\":15,\"i\":\"f19bd0df-03e7-4181-9adf-e3b3b4c97e19\"},\"panelIndex\":\"f19bd0df-03e7-4181-9adf-e3b3b4c97e19\",\"embeddableConfig\":{\"enhancements\":{}},\"panelRefName\":\"panel_f19bd0df-03e7-4181-9adf-e3b3b4c97e19\"},{\"type\":\"map\",\"gridData\":{\"x\":0,\"y\":15,\"w\":24,\"h\":15,\"i\":\"26f25c71-2b7c-4540-8c1c-c003b5657978\"},\"panelIndex\":\"26f25c71-2b7c-4540-8c1c-c003b5657978\",\"embeddableConfig\":{\"mapCenter\":{\"lat\":19.94277,\"lon\":0,\"zoom\":1.4},\"mapBuffer\":{\"minLon\":-180,\"minLat\":-66.51326,\"maxLon\":180,\"maxLat\":66.51326},\"isLayerTOCOpen\":true,\"openTOCDetails\":[],\"hiddenLayers\":[],\"enhancements\":{}},\"panelRefName\":\"panel_26f25c71-2b7c-4540-8c1c-c003b5657978\"},{\"type\":\"search\",\"gridData\":{\"x\":24,\"y\":15,\"w\":24,\"h\":15,\"i\":\"d7e33257-de57-40cb-9171-3dd739dd1875\"},\"panelIndex\":\"d7e33257-de57-40cb-9171-3dd739dd1875\",\"embeddableConfig\":{\"enhancements\":{}},\"panelRefName\":\"panel_d7e33257-de57-40cb-9171-3dd739dd1875\"}]","timeRestore":false,"title":"Managed dashboard with by-ref panels","version":1},"coreMigrationVersion":"8.8.0","created_at":"2024-02-01T22:06:58.824Z","id":"c44c86f9-b105-4a9c-9a24-449a58a827f3","managed":true,"references":[{"id":"managed-feb9-4ba6-9538-1b8f67fb4f57","name":"536c52e4-ecf1-4cde-9323-cc1c3de1fdd2:panel_536c52e4-ecf1-4cde-9323-cc1c3de1fdd2","type":"visualization"},{"id":"managed-36db-4a3b-a4ba-7a64ab8f130b","name":"f19bd0df-03e7-4181-9adf-e3b3b4c97e19:panel_f19bd0df-03e7-4181-9adf-e3b3b4c97e19","type":"lens"},{"id":"managed-d7ab-46eb-a807-8fed28ed8566","name":"26f25c71-2b7c-4540-8c1c-c003b5657978:panel_26f25c71-2b7c-4540-8c1c-c003b5657978","type":"map"},{"id":"managed-3d62-4113-ac7c-de2e20a68fbc","name":"d7e33257-de57-40cb-9171-3dd739dd1875:panel_d7e33257-de57-40cb-9171-3dd739dd1875","type":"search"}],"type":"dashboard","typeMigrationVersion":"8.9.0","updated_at":"2024-02-01T22:06:58.824Z","version":"WzQyMywxXQ=="}
{"excludedObjects":[],"excludedObjectsCount":0,"exportedCount":6,"missingRefCount":0,"missingReferences":[]}

Checklist

Delete any items that are not applicable to this PR.

• Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
• Documentation was added for features that require explanation or tutorials — will happen in [Docs] Add documentation for managed content #175150
• Unit or functional tests were updated or added to match the most common scenarios
• Flaky Test Runner was used on any tests changed — https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5031
• Any UI touched in this PR is usable by keyboard only (learn more about keyboard accessibility)

[drewdaemon]

/ci

[drewdaemon]

/ci

[drewdaemon]

/ci

[lukasolson]

DataDiscovery code changes LGTM, though it wouldn't hurt to get a second pair of eyes from someone a little more familiar with these areas.

[davismcphee]

Code-only review, just serving as a second set of eyes for @lukasolson. The Data Discovery changes LGTM overall. I left a couple small pieces of feedback, but nothing I'd consider blocking or anything.

[ThomThomson]

Changes LGTM! Tested by cloning a managed Dashboard, and by adding managed panels to an unmanaged Dashboard, and checking in the SO Management screen that all of the expected references are present.

Nice work!

[ThomThomson]

Nice generics here! Thanks for the suggestion @davismcphee.

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: e61ea51

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
embeddable	105	106	+1

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
embeddable	459	462	+3

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
dashboard	385.7KB	386.1KB	+341.0B
embeddable	2.0KB	2.3KB	+318.0B
lens	1.4MB	1.4MB	-2.7KB
maps	2.9MB	2.9MB	-1.0KB
savedObjectsFinder	5.0KB	5.2KB	+146.0B
total			-3.0KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
embeddable	65.4KB	65.6KB	+202.0B
lens	41.9KB	45.0KB	+3.1KB
maps	47.9KB	49.1KB	+1.2KB
savedSearch	11.2KB	11.5KB	+271.0B
visualizations	57.4KB	57.6KB	+225.0B
total			+4.9KB

Unknown metric groups

API count

id	before	after	diff
embeddable	564	567	+3

History

• 💚 Build #191400 succeeded 49d657c
• 💚 Build #191069 succeeded bf1d8c8
• 💚 Build #190939 succeeded 7166f52
• 💔 Build #190907 failed 76d31a5
• 💔 Build #190904 failed 72afdf2

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[stratoula]

Looks fantastic my dear Drew! LGTM 🎉