Make SDK update prompt less noisy/more meaningful

[HazAT]

Problem Statement

The problem is that we prompt our users to upgrade their SDK version for every update we ship for SDKs.
We ship a lot of updates so that the prompt could be perceived as unimportant but it shouldn't be.
Also, upgrading without a reason also doesn't make sense; we don't do it, and why should others?

Solution Brainstorm

I am already going into proposals to make these prompts more meaningful.
(Consider it as a suggestion and not how we actually should solve it)

We add a manually maintained entry in the release registry for certain important SDK updates:
https://github.com/getsentry/sentry-release-registry/blob/master/packages/npm/%40sentry/browser/7.8.1.json#L7
To start, we can just document how to do it, and SDK developers should be aware of it. We don't need a complex system around it since it anyway shouldn't happen that often.

e.g.:

"update_text": "{version} contains important security updates"

"update_text": "{version} support new Dynamic Sampling feature, read more: {link}"

We only prompt users if such a text exists and their used version is smaller.
Once dismissed, we never prompt the user again until we ship a new version that again contains update_text

Edit:
ref
https://github.com/getsentry/sentry/blob/0a8a328d9af5fb01217f037d502b6377761ac6de/static/app/components/globalSdkUpdateAlert.tsx

[getsentry-release]

Routing to @getsentry/team-web-sdk-frontend for triage. ⏲️

[getsentry-release]

Routing to @getsentry/team-web-sdk-backend for triage. ⏲️

[getsentry-release]

Routing to @getsentry/team-web-sdk-backend for triage. ⏲️

[silent1mezzo]

👍 love this

[AbhiPrasad]

Another addendum to make this more product driven! Instead of blanket just displaying this update message on every page, we specifically ask for upgrades to unlock certain features in the SDK. For example, we ask folks to upgrade to an SDK that has DS support. We can do this by (finally) introducing an SDK feature list as per getsentry/sentry-release-registry#39. We would introduce a features.json that lives at the root of every SDK repo to make this happen.

We can start off by making update_text live in that feature file, and that's what craft would read to update the release registry accordingly. This leaves us room to expand into a proper SDK feature list.

[jeremybourque]

I love the update text idea too! Couple of suggestions...

• If a user doesn't dismiss the update text after we've shown it to them a few times can we auto-dismiss it for that user so it doesn't linger in the UI needlessly?
• If a new update text appears and the user hasn't installed previous updates, could roll up all the update texts after their current version, or at least the most recent ones, so they know what they're getting?

[souredoutlook]

To add to @AbhiPrasad's ideas:

If we could limit messaging so that the upgrade prompt is only shown for SDK features that the organization's plan supports that would also help limit noise in-app.

[danielkhan]

• If a user doesn't dismiss the update text after we've shown it to them a few times can we auto-dismiss it for that user so it doesn't linger in the UI needlessly?

I may deliberately not dismiss it for a while because I know this needs to be done and want to keep the reminder but suddenly it's gone and I don't know why. Was it retracted? Did someone else do the update?

• If a new update text appears and the user hasn't installed previous updates, could roll up all the update texts after their current version, or at least the most recent ones, so they know what they're getting?

+1

[AbhiPrasad]

The web sdk team is going to be taking a look at this - we'll first start off by writing a DACI with some of the options presented here so we can get consensus on the general approach. This will involve changes to craft, the release registry, the sentry frontend/backend, and possibly the sdks.

[AbhiPrasad]

We've chatted a little about this internally, summarizing those thoughts here:

First we'll be looking through the actual update code in Sentry to see if there are any quick wins we can make, even before introducing some kind of update text metadata.

• Can we improve how "smart" the update banner is without having to change the SDKs/Craft or add a new process?
  • Can we make things focused more toward specific projects so it's more actionable ?
  • Can we change the copy to make this easier to understand?
  • Does increasing the version threshold (for ex: displaying the banner when they are 3 minors behind instead of 1) make the UX better?

Then we'll look at adding some kind of update text to signify important releases.

• Can we just make this a hard coded list to start? Does that mean we have to "backfill" previous important releases? Where does this hard coded list live, sentry or the release registry? If it's the release registry - how do we make sure that all SDK devs are aware of this process? Will that be an extra maintenance burden?
• Can we use craft to make this process easier? Does adding update text live in the release action, or in the SDK repo as some kind of json file. How do we deal with the backfill issue?

[priscilawebdev]

The telemetry experience team is taking up the UX investigation and will follow up with the SDK teams if/when needed.

[nanoburd]

@alexjillard if you can add a link to the change we made. Currently the Performance UX no longer prompts people to update their SDKs, however we still need a flag from the SDK team to trigger the banner to appear under what we feel are "critical" updates.

Critical:

• security
• older SDK versions will be missing data or have problems with data quality

We also need a description of why this is a critical update that customers should be aware of. Beyond a banner a customer communications is probably prudent as well.

Critical is not unlocking new features and this is more of a marketing CTA.

[scefali]

So we turned these prompts off for now because they are too noisy. We are looking into making improvements and bringing this back, however.

[HazAT]

It would be cool if somewhere, in Sentry, there is a small page that shows for your whole org - somewhat like a health check -
All these SDKs need to be updated and could be upgraded.
Sales Engineers use this when working with customers and suggest actions to take.