Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial server-side support for sharing saved-objects phase 1.5 #66089

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
72 commits
Select commit Hold shift + click to select a range
278d345
Add error attribute to SavedObject.error type, update unit tests
jportner Apr 25, 2020
9ee94b2
Modify saved object import function contracts to use type registry
jportner Apr 25, 2020
8c4ec16
Add optional `originId` field to saved objects
jportner Apr 25, 2020
c4fa88a
Change saved object export to omit `namespaces` field
jportner Apr 25, 2020
c1735a6
Modify `bulkCreate` to differentiate between conflict types
jportner Apr 25, 2020
519c260
Add `rawSearchFields` option to `SavedObjectsClient.find` function
jportner Apr 25, 2020
7835833
Validate import objects and retries for uniqueness
jportner May 5, 2020
9e87a9a
Server-side support for importing multi-namespace saved objects
jportner May 5, 2020
924ef96
Server-side support for "duplicate" resolution of import conflicts
jportner May 6, 2020
3741526
Fix existing Spaces API integration tests and refactor
jportner May 7, 2020
a49f114
Add and update API integration tests for Saved Objects and Spaces
jportner May 11, 2020
8ab0a11
API docs changes
jportner May 12, 2020
2d84ae2
Add `originId` field to default saved object mappings
jportner May 13, 2020
7954483
Add `destinationId?` to `SavedObjectsImportConflictError` type
jportner May 19, 2020
01ef8bd
Merge branch 'security/sharing-saved-objects-1.5' into pr/jportner/66089
jportner Jun 2, 2020
f9cb833
Remove "duplicate" resolution of import conflicts
jportner Jun 2, 2020
69eba2e
Merge branch 'security/sharing-saved-objects-1.5' into pr/jportner/66089
jportner Jun 3, 2020
5614bbe
Merge branch 'security/sharing-saved-objects-1.5' into pr/jportner/66089
jportner Jun 8, 2020
ab46928
Merge branch 'security/sharing-saved-objects-1.5' into pr/jportner/66089
jportner Jun 12, 2020
0f273e5
Merge branch 'security/sharing-saved-objects-1.5' into pr/jportner/66089
jportner Jun 15, 2020
19bdeea
Address first round of PR review feedback
jportner Jun 15, 2020
16fc13a
Add `checkConflicts` API to Saved Objects Client
jportner Jun 18, 2020
1dfe7eb
Rename `newId` to `destinationId`
jportner Jun 18, 2020
bb2c7d3
Suppress "ambiguous source" conflict errors
jportner Jun 18, 2020
c9b5391
Merge branch 'master' into HEAD
jportner Jun 26, 2020
76885b2
Unskip tests
jportner Jun 23, 2020
f8776d1
Rename `import/check_conflicts` sub-module
jportner Jun 23, 2020
d65912b
Change import APIs to check for conflicts before creating objects
jportner Jun 24, 2020
4d57561
Fix mock for `find` results
jportner Jun 26, 2020
26db438
Add "true copy" mode for imports
jportner Jun 26, 2020
9289674
Fix `copyToSpace` integration test
jportner Jun 27, 2020
6496c25
Add "true copy" mode for resolving import errors
jportner Jun 28, 2020
0bae0c9
Change `idToOverwrite` field on retry object to `destinationId`
jportner Jun 28, 2020
061f4f7
Merge branch 'security/sharing-saved-objects-1.5' into pr/jportner/66089
jportner Jun 29, 2020
2cbdb75
Clean up Jest integration tests
jportner Jun 29, 2020
f5c56cf
Change outbound reference IDs when import IDs are regenerated
jportner Jun 29, 2020
d30bdfe
Make Jest integration test more realistic
jportner Jun 29, 2020
7badeac
Change import APIs to not create any objects until errors resolved
jportner Jun 29, 2020
af9db43
Merge branch 'security/sharing-saved-objects-1.5' into pr/jportner/66089
jportner Jun 30, 2020
5e70f38
Fix return conflict errors when importing with unresolved errors
jportner Jun 30, 2020
47590a6
Added `trueCopy` field to import success results and retries
jportner Jun 30, 2020
4a1aec9
Merge branch 'master' into pr/jportner/66089
jportner Jul 7, 2020
19dc294
Fix CI errors after merge
jportner Jul 7, 2020
ad92b1a
Change import API's `importIdMap`
jportner Jul 7, 2020
e45adab
True copy: regenerate IDs for resolved missing_reference errors
jportner Jul 7, 2020
25cc72a
Address 2nd round of reviews / nits
jportner Jul 7, 2020
51e05f5
Rename `trueCopy` mode to `createNewCopies`
jportner Jul 7, 2020
b255efb
Clean up copy_to_spaces tests
jportner Jul 8, 2020
45b37c6
Fix integration tests that broke due to `trueCopy` rename
jportner Jul 8, 2020
3538c2b
Change CTS to omit namespace-agnostic object types before import
jportner Jul 8, 2020
2177c4a
Limit concurrent search requests
jportner Jul 8, 2020
3c632db
Updated dev docs for Import and Copy API routes
jportner Jul 8, 2020
0447690
Merge branch 'security/sharing-saved-objects-1.5' into pr/jportner/66089
jportner Jul 13, 2020
dd25fbc
Add inline links from Import docs to Spaces docs
jportner Jul 13, 2020
ba47780
Fix jest test that failed due to recent merge
jportner Jul 13, 2020
5245b66
Merge branch 'security/sharing-saved-objects-1.5' into pr/jportner/66089
jportner Jul 14, 2020
6ebf12a
Fix functional test failures after merge
jportner Jul 14, 2020
48feb3e
Fix type check
jportner Jul 14, 2020
8ec0e37
Fix unit test... (╯°□°)╯︵ ┻━┻
jportner Jul 14, 2020
6136bb0
Merge branch 'security/sharing-saved-objects-1.5' into pr/jportner/66089
jportner Jul 15, 2020
3303559
Rename `rawSearchFields` to `rootSearchFields`
jportner Jul 15, 2020
90bb5a9
Address most nits from fourth review
jportner Jul 15, 2020
ced0415
Remove deprecations
jportner Jul 15, 2020
3570482
Refactor parameters of import modules
jportner Jul 15, 2020
6a67e37
`@ts-ignore` -> `@ts-expect-error`
jportner Jul 15, 2020
ee8a6aa
Merge branch 'security/sharing-saved-objects-1.5' into pr/jportner/66089
jportner Jul 16, 2020
9d7b755
Address fifth round of feedback
jportner Jul 16, 2020
842b0df
Merge branch 'security/sharing-saved-objects-1.5' into pr/jportner/66089
jportner Jul 23, 2020
8cbdc7c
Merge branch 'master' into pr/jportner/66089
jportner Jul 30, 2020
077d52b
Fix type check
jportner Jul 31, 2020
3133deb
Merge branch 'security/sharing-saved-objects-1.5' into pr/jportner/66089
jportner Aug 10, 2020
7e9736c
Merge branch 'security/sharing-saved-objects-1.5' into pr/jportner/66089
jportner Aug 17, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
162 changes: 152 additions & 10 deletions docs/api/saved-objects/import.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,22 @@ experimental[] Create sets of {kib} saved objects from a file created by the exp
==== Path parameters

`space_id`::
(Optional, string) An identifier for the space. If `space_id` is not provided in the URL, the default space is used.
(Optional, string) An identifier for the <<xpack-spaces,space>>. If `space_id` is not provided in the URL, the default space is used.

[[saved-objects-api-import-query-params]]
==== Query parameters

`createNewCopies`::
(Optional, boolean) Creates new copies of saved objects, regenerating each object's ID and resetting its origin in the process. If this
option is used, potential conflict errors will be avoided.
+
NOTE: This cannot be used with the `overwrite` option.

`overwrite`::
(Optional, boolean) Overwrites saved objects.
(Optional, boolean) Overwrites saved objects if they already exist. If this option is used, potential conflict errors will be
automatically resolved by overwriting the destination object.
+
NOTE: This cannot be used with the `createNewCopies` option.
jportner marked this conversation as resolved.
Show resolved Hide resolved

[[saved-objects-api-import-request-body]]
==== Request body
Expand All @@ -37,22 +46,77 @@ The request body must include the multipart/form-data type.
==== Response body

`success`::
Top-level property that indicates if the import was successful.
(boolean) Indicates if the import was completely successful. When set to `false`, some objects may have been copied. For additional
information, refer to the `errors` and `successResults` properties.

`successCount`::
Indicates the number of successfully imported records.
(number) Indicates the number of successfully imported records.

`errors`::
(array) Indicates the import was unsuccessful and specifies the objects that failed to import.

`successResults`::
(array) Indicates the objects that were imported successfully, with any metadata if applicable.
+
NOTE: No objects are actually created until all resolvable errors have been addressed! This includes conflict errors and missing references
errors. See the <<saved-objects-api-resolve-import-errors,Resolve import errors API>> documentation for more information.

[[saved-objects-api-import-codes]]
==== Response code

`200`::
Indicates a successful call.

[[saved-objects-api-import-example]]
==== Examples
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: none of these import examples work out-of-the-box. When only including the minimal "title" attribute, migrations fail. For instance, for Example 3 below I had to change the NDJSON to the following to get it to work:

{"type":"index-pattern","id":"my-pattern","attributes":{"title":"my-pattern-*"}}
{"type":"visualization","id":"my-vis","attributes":{"title":"Look at my visualization"},"migrationVersion":{"visualization":"7.8.0"}}
{"type":"canvas-workpad","id":"my-canvas","attributes":{"name":"Look at my canvas"},"migrationVersion":{"canvas":"7.8.0"}}
{"type":"dashboard","id":"my-dashboard","attributes":{"title":"Look at my dashboard"},"migrationVersion":{"dashboard":"7.3.0"}}

However, I think what we have in the docs already is probably enough, and I am hesitant to add too much more detail. If someone's developing against Kibana they can hopefully figure it out 🙂 I just wanted to point it out here to others.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for raising this, I do think this is something we should address at some point. I personally find it very frustrating when documentation examples don't work.


[[saved-objects-api-import-example-1]]
===== 1. Successful import (with `createNewCopies` enabled)

Import an index pattern and dashboard:

[source,sh]
--------------------------------------------------
$ curl -X POST api/saved_objects/_import?createNewCopies=true -H "kbn-xsrf: true" --form file=@file.ndjson
--------------------------------------------------
// KIBANA

The `file.ndjson` file contains the following:

[source,sh]
--------------------------------------------------
{"type":"index-pattern","id":"my-pattern","attributes":{"title":"my-pattern-*"}}
{"type":"dashboard","id":"my-dashboard","attributes":{"title":"Look at my dashboard"}}
--------------------------------------------------

The API returns the following:

[source,sh]
--------------------------------------------------
{
"success": true,
"successCount": 2,
"successResults": [
{
"id": "my-pattern",
"type": "index-pattern",
"destinationId": "4aba3770-0d04-45e1-9e34-4cf0fd2165ae"
},
{
"id": "my-dashboard",
"type": "dashboard",
"destinationId": "c31d1eca-9bc0-4a81-b5f9-30c442824c48"
}
]
}
--------------------------------------------------

This result indicates that the import was successful, and both objects were created. Since these objects were created as new copies, each
entry in the `successResults` array includes a `destinationId` attribute.

[[saved-objects-api-import-example-2]]
===== 2. Successful import (with `createNewCopies` disabled)

Import an index pattern and dashboard:

[source,sh]
Expand All @@ -75,11 +139,26 @@ The API returns the following:
--------------------------------------------------
{
"success": true,
"successCount": 2
"successCount": 2,
"successResults": [
{
"id": "my-pattern",
"type": "index-pattern"
},
{
"id": "my-dashboard",
"type": "dashboard"
}
]
}
--------------------------------------------------

Import an index pattern and dashboard that includes a conflict on the index pattern:
This result indicates that the import was successful, and both objects were created.

[[saved-objects-api-import-example-3]]
===== 3. Failed import (with conflict errors)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I figured it would be best to collapse all of the different "conflict" errors into one example. This doc is verbose enough as it is.


Import an index pattern, visualization, canvas, and dashboard, where some objects already exists:

[source,sh]
--------------------------------------------------
Expand All @@ -92,6 +171,8 @@ The `file.ndjson` file contains the following:
[source,sh]
--------------------------------------------------
{"type":"index-pattern","id":"my-pattern","attributes":{"title":"my-pattern-*"}}
{"type":"visualization","id":"my-vis","attributes":{"title":"Look at my visualization"}}
{"type":"canvas-workpad","id":"my-canvas","attributes":{"name":"Look at my canvas"}}
{"type":"dashboard","id":"my-dashboard","attributes":{"title":"Look at my dashboard"}}
--------------------------------------------------

Expand All @@ -109,13 +190,71 @@ The API returns the following:
"title": "my-pattern-*",
"error": {
"type": "conflict"
},
}
},
{
"id": "my-visualization",
"type": "my-vis",
"title": "Look at my visualization",
"error": {
"type": "conflict",
"destinationId": "another-vis"
}
},
{
"id": "my-canvas",
"type": "canvas-workpad",
"title": "Look at my canvas",
"error": {
"type": "ambiguous_conflict",
"destinations": [
{
"id": "another-canvas",
"title": "Look at another canvas",
"updatedAt": "2020-07-08T16:36:32.377Z"
},
{
"id": "yet-another-canvas",
"title": "Look at yet another canvas",
"updatedAt": "2020-07-05T12:29:54.849Z"
}
]
}
}
],
"successResults": [
{
"id": "my-dashboard",
"type": "dashboard"
}
]
}
--------------------------------------------------

Import a visualization and dashboard with an index pattern for the visualization reference that doesn't exist:
This result indicates that the import was not successful because the index pattern, visualization, and dashboard each resulted in a conflict
error:

* An index pattern with the same ID already exists, so this resulted in a conflict error. This can be resolved by overwriting the existing
object, or skipping this object entirely.

* A visualization with a different ID but the same "origin" already exists, so this resulted in a conflict error as well. The
`destinationId` field contains the `id` of the other visualization which caused this conflict. This behavior was added to ensure that new
objects which can be shared between <<xpack-spaces,spaces>> behave in a similar way as legacy non-shareable objects. When a shareable object
is exported and then imported into a new space, it retains its origin so that its conflicts will be encountered as expected. This can be
resolved by overwriting the specified destination object, or skipping this object entirely.

* Two canvases with different IDs but the same "origin" already exist, so this resulted in an ambiguous conflict error. The `destinations`
array describes to the other canvases which caused this conflict. When a shareable object is exported and then imported into a new space,
and is _then_ shared to another space where an object of the same origin exists, this situation may occur. This can be resolved by picking
one of the destination objects to overwrite, or skipping this object entirely.

No objects will be imported until this error is resolved using the <<saved-objects-api-resolve-import-errors-example-1,Resolve import errors
API>>.

[[saved-objects-api-import-example-4]]
===== 4. Failed import (with missing reference errors)

Import a visualization and dashboard with an index pattern for the visualization reference that doesn\'t exist:

[source,sh]
--------------------------------------------------
Expand All @@ -127,7 +266,7 @@ The `file.ndjson` file contains the following:

[source,sh]
--------------------------------------------------
{"type":"visualization","id":"my-vis","attributes":{"title":"my-vis"},"references":[{"name":"ref_0","type":"index-pattern","id":"my-pattern-*"}]}
{"type":"visualization","id":"my-vis","attributes":{"title":"Look at my visualization"},"references":[{"name":"ref_0","type":"index-pattern","id":"my-pattern-*"}]}
{"type":"dashboard","id":"my-dashboard","attributes":{"title":"Look at my dashboard"},"references":[{"name":"ref_0","type":"visualization","id":"my-vis"}]}
--------------------------------------------------

Expand All @@ -141,7 +280,7 @@ The API returns the following:
{
"id": "my-vis",
"type": "visualization",
"title": "my-vis",
"title": "Look at my visualization",
"error": {
"type": "missing_references",
"references": [
Expand All @@ -160,3 +299,6 @@ The API returns the following:
}
]
--------------------------------------------------

This result indicates that the import was not successful because the visualization resulted in a missing references error. No objects will
be imported until this error is resolved using the <<saved-objects-api-resolve-import-errors-example-2,Resolve import errors API>>.
Loading