Skip to content
This repository has been archived by the owner on Mar 16, 2021. It is now read-only.

[Package Renames] Expose popularity transfers in search #774

Merged
merged 4 commits into from
May 5, 2020

Conversation

loic-sharma
Copy link
Contributor

@loic-sharma loic-sharma commented May 1, 2020

This exposes the popularity transfer data on the /search/query endpoint. It also exposes the popularity transfer metadata on the /search/diag endpoint. This will later be used to monitor popularity transfer's staleness.

I also introduced a new type, PopularityTransferData to remove the usage of SortedDictionary<string, SortedSet<string>> and remove the need to check case insensitivity for popularity transfers.

Addresses NuGet/NuGetGallery#7903

Add tests for popularity transfer data client

Formatting

Add comment

Clean up

Add test
{
_telemetryClient.TrackMetric(
Prefix + "ReadLatestVerifiedPackagesSeconds",
elapsed.TotalSeconds,
new Dictionary<string, string>
{
{ "PackageIdCount", packageIdCount?.ToString() },
{ "NotModified", notModified.ToString() },
{ "Modified", modified.ToString() },
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This property's name was incorrect. See:

bool modified;
var data = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
AuxiliaryFileMetadata metadata;
try
{
using (var stream = await blobReference.OpenReadAsync(accessCondition))
{
ReadStream(stream, id => data.Add(stringCache.Dedupe(id)));
modified = true;
metadata = new AuxiliaryFileMetadata(
lastModified: new DateTimeOffset(blobReference.LastModifiedUtc, TimeSpan.Zero),
loadDuration: stopwatch.Elapsed,
fileSize: blobReference.Properties.Length,
etag: blobReference.ETag);
}
}
catch (StorageException ex) when (ex.RequestInformation.HttpStatusCode == (int)HttpStatusCode.NotModified)
{
_logger.LogInformation("The blob {BlobName} has not changed.", blobName);
modified = false;
data = null;
metadata = null;
}
stopwatch.Stop();
_telemetryService.TrackReadLatestVerifiedPackages(data?.Count, modified, stopwatch.Elapsed);

using NuGet.Services.AzureSearch.AuxiliaryFiles;

namespace NuGet.Services.AzureSearch.SearchService
{
public class AuxiliaryData : IAuxiliaryData
{
private static readonly string[] EmptyStringArray = new string[0];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I discovered Array.Empty<T>() recently. Seems nice.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice find, I like removing boilerplate :D

@loic-sharma loic-sharma merged commit 91f1ac9 into dev May 5, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants