Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document popularity transfers #2490

Merged
merged 15 commits into from
Aug 17, 2021
Prev Previous commit
Next Next commit
Work
  • Loading branch information
loic-sharma authored Jul 23, 2021
commit 38f98b8c80fc1919a3b3aeb71bb7f6410043f4e1
23 changes: 16 additions & 7 deletions docs/nuget-org/Deprecate-packages.md
Original file line number Diff line number Diff line change
@@ -64,17 +64,16 @@ Project `My.Test.Project` has the following deprecated packages

## Transfer popularity to a newer package

Package authors who have deprecated a legacy package in favor of a newer package can choose to transfer the "popularity" of the legacy to the newer package to boost the its search ranking.
Package authors who have deprecated a legacy package can choose to transfer its "popularity" to a newer package to boost its search ranking. This helps customers discover the newer package instead of the deprecated package.

A common problem that package authors experience when they deprecate a legacy package in favor of a newer package is that the newer, less-popular package will rank lower in search results than the incumbent package for most queries.
For example, let's say I have two packages:

For example, let's say I have two packages for the same task:
* My deprecated legacy package, Contoso.Legacy with 3M downloads
* My latest package, Contoso.Latest with 100K downloads
* My deprecated legacy package, `Contoso.Legacy` with 3 million downloads
* My latest package, `Contoso.Latest` with 100 thousand downloads

NuGet.org heavily considers downloads/popularity when ranking search results. Given the search query "Contoso," Contoso.Legacy will likely rank above Contoso.Latest in the results, making it more difficult for users to discover.
NuGet.org prefers search results with higher downloads/popularity. Given the search query "Contoso", my deprecated package `Contoso.Legacy` would likely rank above my latest package `Contoso.Latest` in search results. However, I'd like my customers to install `Contoso.Latest`!

To solve this problem, I can apply to transfer the popularity of my deprecated legacy package to my latest package. This would cause Contoso.Latest to rank higher in search results as if had Contoso.Legacy's 3M downloads added to its existing 100K downloads. Contoso.Legacy would rank much lower, as if it had almost all of its downloads removed. Only the internal popularity scores for the packages will be impacted, the actual download count shown for each package will not change on NuGet.org or on any client.
To solve this problem, I can apply to transfer the popularity of my deprecated legacy package to my latest package. This would cause `Contoso.Latest` to rank higher in search results, while `Contoso.Legacy` would rank lower. Only the internal popularity scores for the packages is impacted, the actual download count for each package will not be affected.

> [!Note]
> Popularity transfers can make it significantly harder for consumers to find the legacy package.
@@ -92,14 +91,24 @@ See the table below to get a concrete idea of how a popularity transfer may impa

### Popularity transfer application process

1. Review the [popularity transfer requirements](#popularity-transfer-requirements).
2. Determine the deprecated package whose popularity should be transferred. Determine the stable package(s) that should receive the popularity transfer. package you would like to transfer which deprecated packages you would like

After the application is submitted, we will notify you of your application's acceptance or rejection (with the criteria that caused rejection). We may need to ask additional identifying questions to confirm owner identity.

#### Popularity transfer requirements

* The legacy packages and new packages must share all owners.
* The new packages must be clearly related to the legacy packages in naming and function (i.e. an evolution or next generation).
* All versions of the legacy packages must be deprecated and point to the new packages receiving the transfer.
* The popularity transfer must not cause confusion for NuGet users or worsen the NuGet search experience.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This stands out as important to me. Looking at the table above, I can see users being confused by their search results because the actual download counts don't match the order of popularity, leading to mistrust in the nuget.org search results.

Do we have any way of communicating to users that a package's popularity has been transferred - or that a package is special/an outlier in some way? I'm thinking in a subtle, visual way that can be seen at a glance on the search results.

Copy link
Contributor Author

@loic-sharma loic-sharma Jul 26, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the table above, I can see users being confused by their search results because the actual download counts don't match the order of popularity, leading to mistrust in the nuget.org search results.

This is an excellent question. Search looks for results that are most relevant to the user's query. Packages' popularity/downloads is one of many heuristics used to achieve this goal. In other words, it's totally fine for the top results to have low downloads as long as they are the best matches for the search query. If we simply ordered results by their downloads, packages with many downloads would always drown out packages that are most relevant to the user's query.

We did consider a visual cue but decided it wasn't necessary since we only allow popularity transfers if it improves the search results. If a customer were to complain that search results are unexpected, this would be a strong signal that we messed up and allowed a popularity transfer that regresses search rankings.

Here's an example of popularity transfers at work:

image

Notice the top two results have much fewer downloads than the third result. However, the top two results are clearly better than the third!

/cc @chgill-MSFT who drove the spec for popularity transfers

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Loic nailed the explanation exactly. The value of this feature is to actively improve search results in cases where search ranking prioritizing downloads works against the best interest of our users. If we hear a lot of negative customer feedback or see in telemetry that search has been negatively impacted - we'll iterate on the experience accordingly 🙂

* The new packages must have a stable version.
* The legacy package must have fewer than 5 new packages.
loic-sharma marked this conversation as resolved.
Show resolved Hide resolved
* The legacy package must not receive popularity transfers from another deprecated package.

### Advanced popularity transfer scenarios

TODO topics:
* Transfer the popularity of several packages to a consolidated package.
* Transfer the popularity of a single package to multiple separate packages.
* Daisy chaining