User initiated package upgrades

[jonathan-buttner]

User stories

• As an administrator I should be able to see what integrations need to be upgraded.
• As a non administrator I should be able to see that an integration that I'm interested in needs to be upgraded.
• As a SIEM Security solution user I should be able to see that an endpoint package is available for upgrade

Background

The current permissions model makes automatically upgrading packages in the background difficult. A potential work around for this in the short term is to notify the user that an update exists for the packages they’re interested in and provide a mechanism for easily upgrading to the latest version. Users should be encouraged to upgrade to the latest version because it could include bug fixes and new features.

While it is probably unlikely, a user may not navigate back to the Ingest Manager after initially setting it up and installing the necessary packages the first time. For example, a user mostly interested in the SIEM may spend most of their time in that experience. Because of this, the user must be notified about the new package version in the app that relies on the package (e.g. Security Solution app) as well as the Ingest Manager.

Internal packages could be installed automatically if the user has the appropriate privileges but that may be surprising to the user, and the user may want more control over when these upgrades occur. The downside to not automatically upgrading the packages is the user may not install them for a long time.

Notification

Since a user should be notified whether they are in the Ingest Manager app or another app, it probably makes sense to create a reusable component that the Ingest Manager provides to other apps. This component would allow an app to set which packages it is interested in so that only a subset of packages are indicated as being available to upgrade.

The user should probably have the ability to dismiss the notification for a certain amount of time. The notification should come back to remind a user because we should encourage users to upgrade to the most recent version to provide additional features and bug fixes.

Permissions

The current user seeing the notification may not have the appropriate permissions to install the latest package. When the notification is displayed it may be helpful to indicate that an admin needs to log in to perform the upgrade if the current user does not have the correct permissions.

Stack features that rely on package versions

Since there is no guarantee that a stack will have the latest package installed, features that rely on certain package versions being present will need to handle the case when the package is not installed. Ideally long term we could deliver a package as an asset that is installed during the stack upgrade process. This would make certain feature development easier because the feature wouldn’t have to handle the case where the package was not installed.

Notification flow

The diagram below depicts when the notification would be displayed in the Security Solution app. The notification flow in the Ingest Manager app would be somewhat similar but might take the user to a page to display all the packages that are available for upgrade.

The picture below shows the execution flow once a user has initiated upgrading a package.

Related issues

#72714
#64253

[elasticmachine]

Pinging @elastic/endpoint-data-visibility-team (Team:Endpoint Data Visibility)

[elasticmachine]

Pinging @elastic/ingest-management (Team:Ingest Management)

[elasticmachine]

Pinging @elastic/ingest-management (Feature:EPM)

[hbharding]

Hey @jonathan-buttner et al., here are some old mockups and a short screencast of me walking through a Figma prototype I was able to find. It doesn't totally address the "in app" notification pattern you're proposing, but it does provide a way in Ingest Manager to see integrations that have updates available, as well as a button the user can click to update the integration.

Integrations list page with "updates available" filter selected

Integration detail page, settings tab, where user can see the current version # and a button to upgrade (if user has permission)

[hbharding]

As for "in app" notifications, two ideas jump to mind.

1. Use EuiCallout somewhere at the top of the page in the security app to "call out" that updates are available. The callout could include a button which links to the Integration settings page, or perhaps the button in the callout could just trigger an update without having to redirect to Ingest Manager.
2. Trigger a EuiToast when a user navigates to the security app. The toast animates in at the bottom right of the page, and like the callout, it could include a button to link to Ingest Manager. The toast can be dismissed, and may disappear on its own after a certain amount of time has passed.

[hbharding]

One more thing... small improvement for the 2nd diagram you showed.

The idea is that there are 2 possible types of notifications with different messages. The "admin" notification would have a button, whereas the "non admin" notification would not. Basically, don't show the button if the user doesn't have permission to perform the action.

[jonathan-buttner]

Thanks for the notes @hbharding. Yeah your improvement sounds good to me. Any thoughts on reminding the user after they dismiss the notification? Should we do it once a session or something?

[neptunian]

@hbharding I don't think our current UI in Ingest Manager for updating packages is prominent enough to get the user's attention, particularly for packages like Endpoint such as System that we'd prefer to be required auto update but since we can't at the moment, need to the user to do so. Should we have the call out or notification in the Ingest Manager app as well? So we'd be doing the same as the "in app" except listing all the packages that we need them to update.

[hbharding]

Good question. I'm assuming we're talking about an "in app" notification, i.e. something the user would see while in the SIEM app (not ingest manager / fleet). I'd lean towards using EuiCallout since EuiToast will disappear on its own after 10 seconds.

As for dismissing the Callout, this behavior is not built into the component, but it is something we could add. Maybe we could do something like this, which would allow the user to choose when they want to be notified again.

I put this together quickly. We could have a "remind me later" option for non-admins too. Getting the language correct will be important, and we also need to think about what the upgrade button will do. It could trigger an update, or perhaps take the user to the integration settings page.

[ph]

@jonathan-buttner Just to clarify the implementations and who is doing what for this.
@neptunian I will let you confirm.

• We need a way to query the packages that need to be updated.
• Endpoint in their app needs a way to ask for only endpoint integration.
• I suppose we have a component shared between fleet and endpoint app.

@neptunian Do we have everything in place and we only need to wire up the logic?

[jonathan-buttner]

@ph @neptunian yeah I think most of the backend functionality is already there, or might need slight tweaking. For the frontend I agree with Sandy that the Ingest Manager would need:

• A notification or call out that displays all the packages that are available for upgrade when you're in the Ingest Manager
  • Maybe the notification has an Upgrade All button or something?
• A notification or call out that other apps (i.e. Security Solution) can include in their app that will only display a subset of packages that the app is interested in informing the user about. I think this could be 1 or many. For the Security Solution, we'd only be interested in the Endpoint package.
  • Maybe the notification has a button like Henry indicated? Or we could take the user directly to the package's settings page and they have to click on the upgrade button there?

[neptunian]

as @jonathan-buttner said, we need to finalize the user experience and design.

[mostlyjason]

@drewpost @andrewvc would this notification be relevant to uptime when you ship a package upgrade in the future? @jonathan-buttner suggested this could be a reusable component.

[jonathan-buttner]

After our meeting on Monday the decision was made to auto install the default packages in the Ingest Manager and the endpoint package in Security Solution if the user has the appropriate permissions. We're going to hold off on the notification for now.