[Index management] Data stream edit data retention

[sabarasaba]

Fixes: #165136

Summary

This PR adds support for editing data retention for a data stream.

How to test

1. Start elasticsearch with yarn es snapshot --license=trial and serverless search solution with yarn start
2. Navigate to console and execute the commands listed below in order to create a policy
3. Navigate to index management and click on data streams tab
4. Verify that:

• You can edit the data retention, and its reflected in the table and details flyout
• If you set infinite data retention, it should be reflected in the table and details flyout. And when the edit form is open again the toggle should be enabled.
• If infinite toggle is enabled, you shouldnt be allowed to edit data retention values
• If the data stream is managed by ILM, the edit data retention option shouldnt be shown

Creating a test data stream

PUT _index_template/b-index-template
{
  "index_patterns": ["b-data-stream*"],
  "data_stream": { },
  "priority": 500,
  "template": {
    "lifecycle": {
      "enabled": false
    }
  },
  "_meta": {
    "description": "Template with data stream lifecycle"
  }
}

PUT _data_stream/b-data-stream

PUT b-data-stream/_bulk
{ "create":{ } }
{ "@timestamp": "2099-05-06T16:21:15.000Z", "message": "192.0.2.42 - - [06/May/2099:16:21:15 +0000] \"GET /images/bg.jpg HTTP/1.0\" 200 24736" }
{ "create":{ } }
{ "@timestamp": "2099-05-06T16:25:42.000Z", "message": "192.0.2.255 - - [06/May/2099:16:25:42 +0000] \"GET /favicon.ico HTTP/1.0\" 200 3638" }

Screenshots

[sabarasaba]

@elasticmachine merge upstream

[kibanamachine]

merge conflict between base and head

[elasticmachine]

Pinging @elastic/platform-deployment-management (Team:Deployment Management)

[alisonelizabeth]

@elasticmachine merge upstream

[alisonelizabeth]

Really great work @sabarasaba! Thanks for the quick turnaround on this. I tried to test all the different combinations and everything worked as expected (with the known usability issues) - DSL enabled, DSL disabled, DSL enabled with predefined retention, ILM + DSL enabled. Left some comments in the code.

Also, a few other observations:

• I noticed the bug you mentioned to me about "Indices" sometimes showing twice in the flyout. This indeed appears to be a regression of [Index Management] Support data retention on Data Streams tab #165263. I think in data_stream_detail_panel.tsx, if you change L79 to as follows it should fix it:

const descriptionListColumnTwo = descriptionListItems.slice(midpoint, descriptionListItems.length);

• [Follow up] If a user selects "Never delete data" and saves, I think we should represent this in the table view as well (right now this translates as an empty table cell for the "Data retention" column). This is probably more appropriate as a follow-up PR, but just something I noticed and wanted to call out.
• [Follow up] We might want to consider creating human readable units to display in the table. For example 7m could be interpreted as "minutes" or "months". Again, not directly related to your work, just something I noticed.

[florent-leborgne]

Hey @sabarasaba, I'll review the copy today on behalf of @gchaps.

[sabarasaba]

@elasticmachine merge upstream

[sabarasaba]

Thanks for the review @alisonelizabeth! I've addressed your feedback with 59ba5f6 and created a new issue for dealing with the two issues you mentioned above about being more explicit about data retention period in the UI.

[florent-leborgne]

Hey @sabarasaba. I've suggested a few edits. Also have a few questions/remarks for things I'm not sure of:

• I was not sure of what to suggest for the warning, but I think we can make it clearer. See my comments in the review

• Another thing that caught my attention are the shortened time values we use.

  • I understand that they match with the API but "nanos" "micros" etc. sound quite unusual to me compared to common abbreviations (d, h, min, sec, ms, µs, ns) or fully developed forms if we have enough space for it, like 3000 microseconds
  • This maybe also has to do with the API but we render the value with no space in the flyout, while the best practice would be to have a (non-breaking) space between the number and the unit. (30 ns, 4 days, instead of 30ns, 4days). I see it's similar for the storage size so at least it's consistent, but if we can consider changing this at some point in the UI, it would be great.

• Tooltips for "Data retention" differ in the flyout and in the data stream list. I'm not sure if they are still correct, especially the long one? Happy to help revisit them once I'm a bit more sure about the actual behavior ^^'

• Not a content comment, but is it the expected behavior that once we edit the data retention, the flyout also closes?

[sabarasaba]

Thanks for the review @florent-leborgne! I've addressed some of your feedback with 0034336 and I've created this issue to address some of the other smaller things you mentioned.

In regards of the tooltip content, I'm not sure if thats a regresion or not cc: @alisonelizabeth.

[florent-leborgne]

Approving latest copy while noting that pending comments will be addressed through separate issues as discussed. Thanks!

[yuliacech]

Thanks a lot for working on this feature, @sabarasaba!
Left just a couple of nits in the code, but the latest changes LGTM 👍

[sabarasaba]

@elasticmachine merge upstream

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: 3d6e055
• Cloud Deployment

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
indexManagement	586	589	+3

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
indexManagement	192	194	+2

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
indexManagement	553.0KB	560.2KB	+7.1KB
lists	146.9KB	147.0KB	+81.0B
total			+7.2KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
core	367.8KB	367.8KB	+81.0B
indexManagement	33.1KB	33.3KB	+145.0B
total			+226.0B

Unknown metric groups

API count

id	before	after	diff
indexManagement	197	199	+2

History

• 💔 Build #163937 failed 1eb7855
• 💔 Build #163920 failed 8189921
• 💔 Build #163870 failed 70be5ebe64e9299e06976f0f05d51fa0437c7c03
• 💔 Build #163842 failed 9ab511af9b72ca3c50cd59e303e28d9c14f9ddb6
• 💔 Build #163826 failed 59b7d57680c6a96d10a2ff500abd0b39ac172a1b

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @sabarasaba