Unrevert "Migrations v2: don't auto-create indices + FTR/esArchiver support (#85778)" #89992
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…iver support (elastic#85778)"" This reverts commit f979580.
rudolf
added
Team:Core
rudolf
force-pushed
the
reenable-v2-migrations-in-ftr
branch
from
d55ae08 to
1a8e158
Compare
|
@elasticmachine merge upstream |
|
@elasticmachine retest |
|
@elasticmachine merge upstream |
1 similar comment
|
@elasticmachine merge upstream |
|
jenkins test this |
|
@elasticmachine merge upstream |
|
merge conflict between base and head |
rudolf
force-pushed
the
reenable-v2-migrations-in-ftr
branch
from
77eb221 to
c647e18
Compare
|
jenkins test this |
|
last failure seems like it was jenkins related, no tests failed but there was a |
|
jenkins test this |
|
Firefox failure seems to be introduced by https://github.com/elastic/kibana/pull/89942/files#r570906828 |
rudolf
commented
Feb 5, 2021
| @@ -66,20 +66,3 @@ | |||
| } | |||
| } | |||
| } | |||
|
|
|||
| { | |||
There was a problem hiding this comment.
saved objects management sorts by type and the API doesn't allow sorting by two fields. So having a 6 config object and a 8 config object causes unstable tests because of the unstable ordering, so I just deleted this unused object.
jportner
approved these changes
Feb 5, 2021
jbudz
approved these changes
Feb 5, 2021
|
@elasticmachine merge upstream |
|
@elasticmachine merge upstream |
gmmorris
added a commit
to gmmorris/kibana
that referenced
this pull request
Feb 8, 2021
* master: (55 commits) [APM-UI][E2E] use githubNotify step (elastic#90514) [APM] Export ProcessorEvent type (elastic#90540) [Lens] Retain column config (elastic#90048) [Data Table] Add unit tests (elastic#90173) Migrate most plugins to synchronous lifecycle (elastic#89562) skip flaky suite (elastic#90555) skip flaky suite (elastic#64473) [actions] improve email action doc (elastic#90020) [Fleet] Support Fleet server system indices (elastic#89372) skip flaky suite (elastic#90552) Bump immer dependencies (elastic#90267) Unrevert "Migrations v2: don't auto-create indices + FTR/esArchiver support (elastic#85778)" (elastic#89992) [Search Sessions] Use sync config (elastic#90138) chore(NA): add safe guard to remove bazelisk from yarn global at bootstrap (elastic#90538) [test] Await retry.waitFor (elastic#90456) chore(NA): integrate build buddy with our bazel setup and remote cache for ci (elastic#90116) Skip failing suite (elastic#90526) [Fleet] Fix incorrect conversion of string to numeric values in agent YAML (elastic#90371) [Docs] Update reporting troubleshooting for arm rhel/centos (elastic#90385) chore(NA): build bazel projects all at once in the distributable build process (elastic#90328) ...
kibanamachine
added
the
backport missing
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
rudolf
added a commit
to rudolf/kibana
that referenced
this pull request
Feb 9, 2021
…upport (elastic#85778)" (elastic#89992) * Revert "Revert "Migrations v2: don't auto-create indices + FTR/esArchiver support (elastic#85778)"" This reverts commit f979580. * Fix flaky saved objects management test elastic#89953 * If a clone target exists, wait for yellow, not green, index status * Fix test after master merge * Fix types Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
17 tasks
|
Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync. |
2 similar comments
|
Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync. |
|
Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync. |
spalger
added a commit
to spalger/kibana
that referenced
this pull request
Feb 13, 2021
…chiver support (elastic#85778)" (elastic#89992)" This reverts commit fd1d965.
|
Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync. |
💔 Build Failed
Failed CI StepsTest FailuresKibana Pipeline / general / X-Pack API Integration Tests.x-pack/test/api_integration/apis/ml/modules/setup_module·ts.apis Machine Learning modules module setup sets up module data for logs_ui_categories with prefix, startDatafeed true and estimateModelMemory trueStandard OutStack TraceKibana Pipeline / general / X-Pack API Integration Tests.x-pack/test/api_integration/apis/ml/modules/setup_module·ts.apis Machine Learning modules module setup sets up module data for logs_ui_categories with prefix, startDatafeed true and estimateModelMemory trueStandard OutStack TraceMetrics [docs]
History
To update your PR or re-run it, just comment with: |
rudolf
added a commit
that referenced
this pull request
Feb 16, 2021
…iver support (#85778)" (#89992) (#90839) * Unrevert "Migrations v2: don't auto-create indices + FTR/esArchiver support (#85778)" (#89992) * Revert "Revert "Migrations v2: don't auto-create indices + FTR/esArchiver support (#85778)"" This reverts commit f979580. * Fix flaky saved objects management test #89953 * If a clone target exists, wait for yellow, not green, index status * Fix test after master merge * Fix types Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> * Fix visualize fixture after backport * Disable monitoring plugin email notifications in functional tests * Disabling monitoring emails doesnt disable it, configure instead * Fix monitoring test * Remove debug logging from FTR config Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
kibanamachine
removed
the
backport missing
9 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This reverts commit f979580 to add back v2 migrations to FTR after #85778 was reverted.
From #85778
Saved objects no longer auto-creates indices
The new migrations aren't compatible with an auto-created saved objects index, so when tests delete
.kibanaand then use the saved objects client, the next test will fail because migrations fail. We could hack the v2 migrations to still work on the auto-created indices, but this is never the behaviour a user wants. If the saved objects index was deleted while Kibana was running we should "stop the world" and let users investigate. Our current behaviour means users sometimes end up with some changes in a new index and cannot "merge" the two indices and causes all sorts of runtime failures because of incompatible mappings or upgrade failures.So this PR starts implementing the decision in #81790 to never auto-create system indices. I'm using the
require_aliasquery parameter which means we could theoretically miss a route, but something is better than nothing until we move to a system index.It reverses the decision in #14141 where we formalized the auto-create index behaviour in 6.1. However, when this decision was made Kibana used an index template so an auto-created index was acceptable since it would always have the correct mappings. With this PR, if the saved objects index and aliases doesn't exist, indexing actions will return a 500. I think we should also return a 500 for read operations when the index doesn't exist since Kibana really is broken, it's not the same as not being able to find a particular saved object, but I felt like this could be postponed into a follow-up PR (created #89642 to track this).
Enables v2 migrations in all tests
Because of removing the auto-create index behaviour I had to adapt tests to always initialize an empty kibana with
esArchiver.emptyKibanaIndex()or move somebeforestatements around so that we don't set UiSettings before the index gets initialized.Closes #85911
Closes #91445
In this PR
I also changed the migrations algorithm slightly. Before, if a clone or create index failed with
index_already_exists_exceptionwe would wait until that index's status turns green. This PR changes that to only wait for a yellow status.Checklist
Delete any items that are not applicable to this PR.
For maintainers