[ILM] Add support for frozen phase #93068

sebelga · 2021-03-01T17:03:46Z

This PR adds support for the frozen phase and the frozen tier in ILM. The frozen phase is following the same rules as the cold phase with the exception that the frozen phase defaults searchable_snapshot.storage to shared_cache where in cold phase it is full_copy.

License

Frozen phase has only one action: searchable snapshot. As searchable snapshot requires the entreprise license, the frozen phase is only visible for users with that license. In basic license the frozen phase is not shown.

How to test

Create a new ILM policy, the frozen phase should be disabled
Enable the frozen phase
Create searchable snapshot should be disabled. (In cloud it should be enabled) by default
There should be a "More option" button to access the storage option
Make sure the advanced settings work as expected
Check that the "Sow request" correctly serializes the policy and the frozen phase

To simulate that Kibana runs in cloud context, add the following in the kibana.dev.yml

xpack.cloud.id: 'eastus2.azure.elastic-cloud.com:9243$59ef636c6917463db140321484d63cfa$a8b109c08adc43279ef48f29af1a3911'
xpack.cloud.deploymentUrl: 'https://cloud.elastic.co/deployments/resolve/cluster/eastus2/9243$59ef636c6917463db140321484d63cfa/'

Release note

You can now configure the frozen phase in your index lifecycle policy. This new data tier is optimized for maximum cost savings.

elasticmachine · 2021-03-04T12:57:35Z

Pinging @elastic/es-ui (Team:Elasticsearch UI)

jrodewig

Thanks @sebelga! I left some copy suggestions. Let me know if you have any questions.

x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/i18n_texts.ts

...ment/public/application/sections/edit_policy/components/phases/frozen_phase/frozen_phase.tsx

...ck/plugins/index_lifecycle_management/public/application/sections/edit_policy/form/schema.ts

...ifecycle_management/public/application/sections/edit_policy/components/more_less_section.tsx

yuliacech · 2021-03-04T16:56:56Z

...k/plugins/index_lifecycle_management/public/application/sections/edit_policy/edit_policy.tsx

+              <EuiSpacer />
+              <FrozenPhase />
+
+              <EuiSpacer />


Adding a spacer between frozen and delete phases makes the connecting line go beyond frozen phase:

I think this spacer needs to go into delete phase to fix it. There is an issue open in EUI repo to address 'gutter size' between 'comment items' but it is not being worked on currently.

Great catch! 👍

I moved to the DeletePhase and it indeed fixed the issue.

Thanks for fixing this, @sebelga!

yuliacech · 2021-03-04T16:57:52Z

...ifecycle_management/public/application/sections/edit_policy/components/more_less_section.tsx

+  align?: 'left' | 'right';
+}
+
+export const MoreLessSection: FunctionComponent<Props> = ({ align = 'left', children }) => {


I think this component can be replaced with EuiAccordion, WDYT?

I am not a huge fan of the accordion and the animation in this case. I prefer the snappiness of an actual toggle

I see your reasoning here @sebelga, but then having 2 different animations make the UI inconsistent.
Should we change Advanced settings to the new component too?
I also have concerns about built-in a11y features that handle this kind of interaction in accordion. There are attributes like aria-controls, aria-expanded, role, aria-labelledby that we need to implement in the new component as well.

I think Yulia raises great points about accessibility. My advice would be to leverage EUI as much as possible, and then drive changes at the EUI level to support the behavior you want @sebelga. For example, ask for a timing or animation prop to be added to EuiAccordion. This will have the benefit of getting input from the design team, it will share the new behavior to all EUI consumers to reduce the potential for duplication, and it will ensure consistency throughout our UIs.

Big +1 to using EUI components and requesting desired enhancements on their repo.

An additional future consideration here... there is an existing Advanced setting to 'Disable animations' globally. Imagine a future - with user settings - where an individual can toggle this as their personal preference.

I only recently learned of this setting and just now tried it out - as a quick test, it removed the animation from the accordions in the Canvas sidebar.

Good point about accessibility @yuliacech and leverage EUI effort on this, I will replace the component with the accordion 👍

There is an issue with the <EuiAccordion /> that clips the combox helptext after the first line. I've opened an issue for it (elastic/eui#4623). Should we block this PR until the issue is solved?

yuliacech

Hi @sebelga, thanks a lot for working on this!
Left 2 comments in the code, curios what you think.

Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>

...ment/public/application/sections/edit_policy/components/phases/frozen_phase/frozen_phase.tsx

...ts/phases/shared_fields/data_tier_allocation_field/components/missing_cloud_tier_callout.tsx

...cy/components/phases/shared_fields/data_tier_allocation_field/data_tier_allocation_field.tsx

...licy/components/phases/shared_fields/searchable_snapshot_field/searchable_snapshot_field.tsx

yuliacech

Hi @sebelga,
thanks a lot for working on this! I re-tested locally and all worked for me. Left a couple of comments in the code, but nothing blocking.

sebelga · 2021-03-09T10:21:37Z

Thanks for the review @yuliacech ! I addressed your last requests for change 👍

jrodewig

Thanks for reaching out offline @sebelga. I've included some updated copy for the searchable snapshots field descriptions.

@dakrone Let me know if you have any feedback on this copy. Thanks!

Searchable snapshot for hot and cold tiers

Take a snapshot of your data and mount it as a searchable snapshot. <Learn more>.

The <Learn more> link remains the same.

Searchable snapshot for frozen tiers

Take a snapshot of your data and mount it as a searchable snapshot. To reduce costs, only a cache of the snapshot is mounted in the frozen tier. <Learn more>.

<Learn more> should link to https://www.elastic.co/guide/en/elasticsearch/reference/master/searchable-snapshots.html#searchable-snapshots-shared-cache

x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/i18n_texts.ts

…ections/edit_policy/i18n_texts.ts Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>

…lm-ui/frozen-tier

sebelga · 2021-03-10T09:57:50Z

Thanks for the quick update @jrodewig ! Your suggestions look good to me.

note: hot phase has the same description for searchable snapshot as cold phase

kibanamachine · 2021-03-10T20:03:08Z

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`indexLifecycleManagement`	195	198	+3

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`indexLifecycleManagement`	242.0KB	251.0KB	+9.0KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`indexLifecycleManagement`	53.3KB	53.5KB	+133.0B

History

💔 Build #112411 failed 6ef0580
💔 Build #112053 failed fe45a15
💔 Build #112023 failed 30c6b28
💚 Build #111776 succeeded 3f64e8f
💛 Build #111634 was flaky 6878826

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

…-action * 'master' of github.com:elastic/kibana: (43 commits) [Console] Update copy when showing warnings in response headers (#94270) [TSVB] Type public code. Step 1 (#93231) [ML] Functional tests - stabilize slider value selection (#94313) skip another suite blocking es promotion (#94367) [Security Solution] Eliminates a redundant external link icon (#94194) skip another suite blocking es promotion (#94367) [App Search] Role mappings migration part 1 (#94346) [Security Solution][Detections] Fix flaky indicator enrichment tests (#94241) [Workplace Search] Deduplicate icons (#94359) [ML] Add latest transform to intro text (#94039) skip test failing es promotion (#94367) [Maps] convert elasticsearch_utils to TS (#93984) [Security_Solution][Telemetry] - Update endpoint usage to use agentService (#93829) [Security Solution][Exceptions] Fixes OS adding method for exception enrichment (#94343) [ILM] Add support for frozen phase (#93068) [App Search] Fixed 2 relevance tuning bugs (#94312) remove `try` auth mode (#94287) Removing resolver functional tests (#94331) migrate warning mixin to core (#94273) [App Search] Add routes for Role Mappings (#94221) ... # Conflicts: # x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/phases/cold_phase/cold_phase.tsx # x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/phases/phase/phase.tsx # x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/phases/shared_fields/searchable_snapshot_field/searchable_snapshot_field.tsx # x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/edit_policy.tsx # x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/form/configuration_issues_context.tsx

sebelga added Feature:ILM Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more v7.13.0 v8.0.0 labels Mar 1, 2021

sebelga mentioned this pull request Mar 2, 2021

[ILM] Allow multiple searchable snapshot actions #92789

Merged

sebelga force-pushed the ilm-ui/frozen-tier branch from 64cdc5a to 6202e37 Compare March 2, 2021 17:50

sebelga added 10 commits March 4, 2021 11:44

Add frozen phase to UI

33ba507

Add storage option to searchable snapshot

386a455

Update form de/serializer

34103d5

Update server request body validation

df40add

Add more/less section to hide storage parameter

7a9daf5

Update component integration tests

7e0736b

Add attribute key to ignore

560967c

Use loop to test both cold and frozen tiers

44d51a2

Add missing tier callout in cloud for frozen tier

927126d

Update API integration test to include frozen in fixture

6fb98fb

sebelga force-pushed the ilm-ui/frozen-tier branch from 6202e37 to 6fb98fb Compare March 4, 2021 12:08

sebelga marked this pull request as ready for review March 4, 2021 12:57

sebelga requested review from a team as code owners March 4, 2021 12:57

sebelga requested a review from yuliacech March 4, 2021 12:58

sebelga added 2 commits March 4, 2021 13:05

Align "more options" button to the left

03daf2a

Update placeholder for copy text

c0960ef

jrodewig reviewed Mar 4, 2021

View reviewed changes

yuliacech reviewed Mar 4, 2021

View reviewed changes

yuliacech suggested changes Mar 4, 2021

View reviewed changes

sebelga and others added 2 commits March 5, 2021 10:29

Move EuiSpacer inside the delete phase to fix connecting line issue

4ff0ccd

Apply suggestions from code review

a1e5bb3

Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>

sebelga added the release_note:enhancement label Mar 8, 2021

Merge remote-tracking branch 'upstream/master' into ilm-ui/frozen-tier

3f64e8f

yuliacech reviewed Mar 8, 2021

View reviewed changes

...ment/public/application/sections/edit_policy/components/phases/frozen_phase/frozen_phase.tsx Outdated Show resolved Hide resolved

yuliacech reviewed Mar 8, 2021

View reviewed changes

...ts/phases/shared_fields/data_tier_allocation_field/components/missing_cloud_tier_callout.tsx Show resolved Hide resolved

yuliacech reviewed Mar 8, 2021

View reviewed changes

...cy/components/phases/shared_fields/data_tier_allocation_field/data_tier_allocation_field.tsx Outdated Show resolved Hide resolved

yuliacech reviewed Mar 8, 2021

View reviewed changes

...licy/components/phases/shared_fields/searchable_snapshot_field/searchable_snapshot_field.tsx Outdated Show resolved Hide resolved

yuliacech approved these changes Mar 8, 2021

View reviewed changes

sebelga added 2 commits March 9, 2021 09:43

Extract freeze field into reusable component

27bcdb7

Update comments to include the frozen tier

30c6b28

sebelga added 4 commits March 9, 2021 11:48

Fix text clipping issue in accordion

fe45a15

Fix i18n issues

355592f

Don't allow to disable searchable snapshot for frozen phase

859c92c

Remove all advanced settings from frozen phase

21b1340

jrodewig reviewed Mar 9, 2021

View reviewed changes

x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/i18n_texts.ts Outdated Show resolved Hide resolved

sebelga and others added 6 commits March 10, 2021 09:23

Update component integration test

ba08e34

Add phase footer to frozen phase

a0249cd

Merge remote-tracking branch 'upstream/master' into ilm-ui/frozen-tier

5b8a6ba

Update text copy for searchable snapshot

abf907a

Update x-pack/plugins/index_lifecycle_management/public/application/s…

edf95a5

…ections/edit_policy/i18n_texts.ts Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>

Merge branch 'ilm-ui/frozen-tier' of github.com:sebelga/kibana into i…

5f99ba6

…lm-ui/frozen-tier

sebelga added 2 commits March 10, 2021 10:28

Hide frozen phase on non entreprise license

6ef0580

Fix API integration tests

7d5c02b

sebelga merged commit a632f3f into elastic:master Mar 10, 2021

sebelga mentioned this pull request Mar 10, 2021

[7.x] [ILM] Add support for frozen phase (#93068) #94361

Merged

sebelga added a commit that referenced this pull request Mar 10, 2021

[ILM] Add support for frozen phase (#93068) (#94361)

087cff5

cjcenizal mentioned this pull request Apr 8, 2021

Add frozen phase to ILM UI #91189

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ILM] Add support for frozen phase #93068

[ILM] Add support for frozen phase #93068

sebelga commented Mar 1, 2021 •

edited

Loading

elasticmachine commented Mar 4, 2021

jrodewig left a comment •

edited

Loading

yuliacech Mar 4, 2021

sebelga Mar 5, 2021

sebelga Mar 5, 2021

yuliacech Mar 5, 2021

yuliacech Mar 4, 2021

sebelga Mar 5, 2021

yuliacech Mar 5, 2021

cjcenizal Mar 5, 2021

ryankeairns Mar 5, 2021

ryankeairns Mar 5, 2021 •

edited

Loading

sebelga Mar 8, 2021

sebelga Mar 8, 2021

yuliacech left a comment

yuliacech left a comment

sebelga commented Mar 9, 2021

jrodewig left a comment

sebelga commented Mar 10, 2021 •

edited

Loading

kibanamachine commented Mar 10, 2021

[ILM] Add support for frozen phase #93068

[ILM] Add support for frozen phase #93068

Conversation

sebelga commented Mar 1, 2021 • edited Loading

License

How to test

Release note

elasticmachine commented Mar 4, 2021

jrodewig left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ryankeairns Mar 5, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yuliacech left a comment

Choose a reason for hiding this comment

yuliacech left a comment

Choose a reason for hiding this comment

sebelga commented Mar 9, 2021

jrodewig left a comment

Choose a reason for hiding this comment

Searchable snapshot for hot and cold tiers

Searchable snapshot for frozen tiers

sebelga commented Mar 10, 2021 • edited Loading

kibanamachine commented Mar 10, 2021

💚 Build Succeeded

Metrics [docs]

Module Count

Async chunks

Page load bundle

History

sebelga commented Mar 1, 2021 •

edited

Loading

jrodewig left a comment •

edited

Loading

ryankeairns Mar 5, 2021 •

edited

Loading

sebelga commented Mar 10, 2021 •

edited

Loading