Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding basic request diagnostics for V3 #615

Merged
merged 26 commits into from
Aug 30, 2019
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
e7c3424
inital commit
simplynaveen20 Jul 31, 2019
0a0f200
making cosmosDiagnostic public
simplynaveen20 Jul 31, 2019
1b75148
bumping direct version
simplynaveen20 Jul 31, 2019
431de60
Merge branch 'master' into users/nakumar/requestDiagnosticsV3
simplynaveen20 Aug 1, 2019
1478d41
removing formatting of json
simplynaveen20 Aug 1, 2019
464eccf
merging with master
simplynaveen20 Aug 16, 2019
94bf545
removing extra line
simplynaveen20 Aug 16, 2019
34774d3
Merge branch 'master' into users/nakumar/requestDiagnosticsV3
simplynaveen20 Aug 19, 2019
0367e03
updating per new changes with clientsiderequestdiagnostics
simplynaveen20 Aug 19, 2019
bd317ad
Merge branch 'master' into users/nakumar/requestDiagnosticsV3
simplynaveen20 Aug 21, 2019
567521a
using to string on CLientSideRequestDiagnostics
simplynaveen20 Aug 21, 2019
06613ae
updating DOtNetSDKAPI
simplynaveen20 Aug 21, 2019
feaef5e
updating DotNetSDKAPI.json
simplynaveen20 Aug 21, 2019
179281f
moving cosmosDiagnostic as abstract clas
simplynaveen20 Aug 21, 2019
4d82e31
adding abstract tostring
simplynaveen20 Aug 22, 2019
8fb181c
updating change log
simplynaveen20 Aug 22, 2019
baace6c
removing query metrics from header
simplynaveen20 Aug 22, 2019
af0b757
removing populateQueryOption flag from request option
simplynaveen20 Aug 26, 2019
04855e7
merging with master
simplynaveen20 Aug 27, 2019
9ce1424
updating contract
simplynaveen20 Aug 27, 2019
250d43e
Merge branch 'master' into users/nakumar/requestDiagnosticsV3
simplynaveen20 Aug 27, 2019
f84aa5e
Merge branch 'master' into users/nakumar/requestDiagnosticsV3
j82w Aug 29, 2019
00dd4c3
upgrading direct to 3.1.6 to leverage IClientSideRequestDiagnostics
simplynaveen20 Aug 29, 2019
ac5b28f
Merge branch 'master' into users/nakumar/requestDiagnosticsV3
simplynaveen20 Aug 30, 2019
cd411e1
changing name of cosmosDiagnostics to diagnostics
simplynaveen20 Aug 30, 2019
9a42e92
moving diagnostics to Diagnostics
simplynaveen20 Aug 30, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions Microsoft.Azure.Cosmos/src/Handler/ResponseMessage.cs
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,11 @@ public virtual Stream Content
/// </summary>
public virtual RequestMessage RequestMessage { get; internal set; }

/// <summary>
/// Gets the cosmos diagnostic information for the current request to Azure Cosmos DB service
/// </summary>
public CosmosDiagnostic cosmosDiagnostic { get; set; }
simplynaveen20 marked this conversation as resolved.
Show resolved Hide resolved

/// <summary>
/// Gets the internal error object.
/// </summary>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ internal CosmosQueryResponseMessageHeaders CloneKnownProperties(
RetryAfterLiteral = this.RetryAfterLiteral,
SubStatusCodeLiteral = this.SubStatusCodeLiteral,
ContentType = this.ContentType,
QueryMetricsText = QueryMetricsText
};
}

Expand Down Expand Up @@ -105,6 +106,7 @@ internal static CosmosQueryResponseMessageHeaders ConvertToQueryHeaders(
RetryAfterLiteral = sourceHeaders.RetryAfterLiteral,
SubStatusCodeLiteral = sourceHeaders.SubStatusCodeLiteral,
ContentType = sourceHeaders.ContentType,
QueryMetricsText = sourceHeaders.QueryMetricsText
};
}
}
Expand Down
3 changes: 3 additions & 0 deletions Microsoft.Azure.Cosmos/src/Headers/Headers.cs
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,9 @@ internal string RetryAfterLiteral
[CosmosKnownHeaderAttribute(HeaderName = HttpConstants.HttpHeaders.PageSize)]
internal string PageSize { get; set; }

[CosmosKnownHeaderAttribute(HeaderName = HttpConstants.HttpHeaders.QueryMetrics)]
internal string QueryMetricsText { get; set; }
simplynaveen20 marked this conversation as resolved.
Show resolved Hide resolved

/// <summary>
/// Creates a new instance of <see cref="Headers"/>.
/// </summary>
Expand Down
2 changes: 1 addition & 1 deletion Microsoft.Azure.Cosmos/src/Microsoft.Azure.Cosmos.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
<Copyright>© Microsoft Corporation. All rights reserved.</Copyright>
<NeutralLanguage>en-US</NeutralLanguage>
<ClientVersion>3.1.0</ClientVersion>
<DirectVersion>3.1.3</DirectVersion>
<DirectVersion>3.1.4</DirectVersion>
<HybridRowVersion>1.0.0-preview</HybridRowVersion>
<DirectPackageName Condition=" '$(SignAssembly)' == 'true' ">Microsoft.Azure.Cosmos.Direct</DirectPackageName>
<DirectPackageName Condition=" '$(SignAssembly)' != 'true' ">Microsoft.Azure.Cosmos.Direct.MyGet</DirectPackageName>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -392,6 +392,7 @@ public override async Task<QueryResponse> DrainAsync(int maxElements, Cancellati
result: results,
count: results.Count,
responseHeaders: this.GetResponseHeaders(),
queryMetrics: this.GetQueryMetrics(),
responseLengthBytes: this.GetAndResetResponseLengthBytes());
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,15 @@ private async Task<ResponseMessage> ExecuteNextHelperAsync(CancellationToken can
}
}

if (response?.queryMetrics != null && response?.queryMetrics.Count > 0)
{
QueryOperationStatistics queryOperationStatistics = new QueryOperationStatistics(response.queryMetrics);
response.cosmosDiagnostic = new CosmosDiagnostic
{
queryOperationStatistics = queryOperationStatistics
};
}

return response;
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,6 @@ public override async Task<QueryResponse> DrainAsync(int maxElements, Cancellati
PartitionedQueryMetrics partitionedQueryMetrics = new PartitionedQueryMetrics();
ResourceType resourceType = ResourceType.Document;
string containerRid = null;

while (!this.IsDone)
simplynaveen20 marked this conversation as resolved.
Show resolved Hide resolved
{
QueryResponse result = await base.DrainAsync(int.MaxValue, token);
Expand Down Expand Up @@ -168,7 +167,8 @@ public override async Task<QueryResponse> DrainAsync(int maxElements, Cancellati
containerRid: containerRid)
{
RequestCharge = requestCharge
});
},
queryMetrics: this.GetQueryMetrics());
}

/// <summary>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,8 @@ public override async Task<QueryResponse> DrainAsync(int maxElements, Cancellati
distinctResults,
distinctResults.Count,
cosmosQueryResponse.ResponseLengthBytes,
cosmosQueryResponse.QueryHeaders.CloneKnownProperties(updatedContinuationToken, disallowContinuationTokenMessage));
cosmosQueryResponse.QueryHeaders.CloneKnownProperties(updatedContinuationToken, disallowContinuationTokenMessage),
cosmosQueryResponse.queryMetrics);
}

/// <summary>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,7 @@ public override async Task<QueryResponse> DrainAsync(int maxElements, Cancellati
result: documentsAfterSkip,
count: documentsAfterSkip.Count(),
responseHeaders: sourcePage.QueryHeaders.CloneKnownProperties(updatedContinuationToken, sourcePage.QueryHeaders.DisallowContinuationTokenMessage),
queryMetrics: sourcePage.queryMetrics,
responseLengthBytes: sourcePage.ResponseLengthBytes);
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,8 @@ public override async Task<QueryResponse> DrainAsync(int maxElements, Cancellati
takedDocuments,
takedDocuments.Count,
results.ResponseLengthBytes,
results.QueryHeaders.CloneKnownProperties(updatedContinuationToken, results.QueryHeaders.DisallowContinuationTokenMessage));
results.QueryHeaders.CloneKnownProperties(updatedContinuationToken, results.QueryHeaders.DisallowContinuationTokenMessage),
results.queryMetrics);
}

private enum TakeEnum
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -387,7 +387,8 @@ public override async Task<QueryResponse> ExecuteNextAsync(CancellationToken tok
dynamics,
queryResponse.Count,
queryResponse.ResponseLengthBytes,
queryResponse.QueryHeaders.CloneKnownProperties());
queryResponse.QueryHeaders.CloneKnownProperties(),
queryMetrics: queryResponse.queryMetrics);
}
catch (Exception)
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,16 @@ public ConsistencyLevel? ConsistencyLevel
set => this.BaseConsistencyLevel = value;
}

/// <summary>
/// Gets or sets the PopulateQueryMetrics request option for item query requests in the Azure Cosmos DB service.
/// </summary>
/// <remarks>
/// <para>
/// PopulateQueryMetrics is used to enable/disable getting metrics relating to query execution on item query requests.
/// </para>
/// </remarks>
public bool PopulateQueryMetrics { get; set; }
simplynaveen20 marked this conversation as resolved.
Show resolved Hide resolved

/// <summary>
/// Gets or sets the token for use with session consistency in the Azure Cosmos DB service.
/// </summary>
Expand Down Expand Up @@ -197,6 +207,11 @@ internal override void PopulateRequestOptions(RequestMessage request)
request.Headers.Add(HttpConstants.HttpHeaders.ContentSerializationFormat, this.CosmosSerializationOptions.ContentSerializationFormat);
}

if (this.PopulateQueryMetrics)
{
request.Headers.Add(HttpConstants.HttpHeaders.PopulateQueryMetrics, bool.TrueString);
}

base.PopulateRequestOptions(request);
}

Expand All @@ -218,7 +233,8 @@ internal QueryRequestOptions Clone()
CosmosSerializationOptions = this.CosmosSerializationOptions,
EnableCrossPartitionSkipTake = this.EnableCrossPartitionSkipTake,
Properties = this.Properties,
IsEffectivePartitionKeyRouting = this.IsEffectivePartitionKeyRouting
IsEffectivePartitionKeyRouting = this.IsEffectivePartitionKeyRouting,
PopulateQueryMetrics = this.PopulateQueryMetrics
};

return queryRequestOptions;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,13 @@ protected ItemResponse()
internal ItemResponse(
HttpStatusCode httpStatusCode,
Headers headers,
T item)
T item,
CosmosDiagnostic cosmosDiagnostic)
{
this.StatusCode = httpStatusCode;
this.Headers = headers;
this.Resource = item;
this.cosmosDiagnostic = cosmosDiagnostic;
}

/// <inheritdoc/>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@
namespace Microsoft.Azure.Cosmos
{
using System;
using System.Net;
using System.Threading.Tasks;
using Microsoft.Azure.Cosmos.Scripts;

Expand Down Expand Up @@ -39,7 +38,7 @@ internal FeedResponse<T> CreateQueryFeedResponse<T>(
if (queryResponse != null)
{
return QueryResponse<T>.CreateResponse<T>(
responseMessage: queryResponse,
cosmosQueryResponse: queryResponse,
jsonSerializer: this.cosmosSerializer);
}

Expand All @@ -57,7 +56,8 @@ internal Task<ItemResponse<T>> CreateItemResponseAsync<T>(
return new ItemResponse<T>(
cosmosResponseMessage.StatusCode,
cosmosResponseMessage.Headers,
item);
item,
cosmosResponseMessage.cosmosDiagnostic);
});
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ private QueryResponse(
int count,
long responseLengthBytes,
CosmosQueryResponseMessageHeaders responseHeaders,
IReadOnlyDictionary<string, QueryMetrics> queryMetrics,
HttpStatusCode statusCode,
RequestMessage requestMessage,
string errorMessage,
Expand All @@ -42,6 +43,7 @@ private QueryResponse(
this.CosmosElements = result;
this.Count = count;
this.ResponseLengthBytes = responseLengthBytes;
this.queryMetrics = queryMetrics;
}

public int Count { get; }
Expand Down Expand Up @@ -72,6 +74,8 @@ private QueryResponse(
/// </remarks>
internal ClientSideRequestStatistics RequestStatistics { get; }

internal IReadOnlyDictionary<string, QueryMetrics> queryMetrics { get; set; }

internal virtual CosmosSerializationOptions CosmosSerializationOptions { get; set; }

internal bool GetHasMoreResults()
Expand All @@ -83,7 +87,8 @@ internal static QueryResponse CreateSuccess(
IEnumerable<CosmosElement> result,
int count,
long responseLengthBytes,
CosmosQueryResponseMessageHeaders responseHeaders)
CosmosQueryResponseMessageHeaders responseHeaders,
IReadOnlyDictionary<string, QueryMetrics> queryMetrics = null)
{
if (count < 0)
{
Expand All @@ -100,6 +105,7 @@ internal static QueryResponse CreateSuccess(
count: count,
responseLengthBytes: responseLengthBytes,
responseHeaders: responseHeaders,
queryMetrics: queryMetrics,
statusCode: HttpStatusCode.OK,
errorMessage: null,
error: null,
Expand All @@ -120,6 +126,7 @@ internal static QueryResponse CreateFailure(
count: 0,
responseLengthBytes: 0,
responseHeaders: responseHeaders,
queryMetrics: null,
statusCode: statusCode,
errorMessage: errorMessage,
error: error,
Expand All @@ -146,11 +153,13 @@ internal class QueryResponse<T> : FeedResponse<T>
private QueryResponse(
IEnumerable<CosmosElement> cosmosElements,
CosmosQueryResponseMessageHeaders responseMessageHeaders,
CosmosDiagnostic cosmosDiagnostic,
CosmosSerializer jsonSerializer,
CosmosSerializationOptions serializationOptions)
{
this.cosmosElements = cosmosElements;
this.QueryHeaders = responseMessageHeaders;
this.cosmosDiagnostic = cosmosDiagnostic;
this.jsonSerializer = jsonSerializer;
this.serializationOptions = serializationOptions;
}
Expand Down Expand Up @@ -211,21 +220,6 @@ public override IEnumerable<T> Resource
}
}

internal static QueryResponse<TInput> CreateResponse<TInput>(
ResponseMessage responseMessage,
CosmosSerializer jsonSerializer)
{
QueryResponse queryResponse = responseMessage as QueryResponse;
if (queryResponse == null)
{
throw new ArgumentException($"{nameof(responseMessage)} must be of type query response.");
}

return QueryResponse<TInput>.CreateResponse<TInput>(
queryResponse,
jsonSerializer);
}

internal static QueryResponse<TInput> CreateResponse<TInput>(
QueryResponse cosmosQueryResponse,
CosmosSerializer jsonSerializer)
Expand All @@ -237,10 +231,10 @@ internal static QueryResponse<TInput> CreateResponse<TInput>(
queryResponse = new QueryResponse<TInput>(
cosmosElements: cosmosQueryResponse.CosmosElements,
responseMessageHeaders: cosmosQueryResponse.QueryHeaders,
cosmosDiagnostic: cosmosQueryResponse.cosmosDiagnostic,
jsonSerializer: jsonSerializer,
serializationOptions: cosmosQueryResponse.CosmosSerializationOptions);
}

return queryResponse;
}
}
Expand Down
5 changes: 5 additions & 0 deletions Microsoft.Azure.Cosmos/src/Resource/Response.cs
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,11 @@ public static implicit operator T(Response<T> response)
/// </remarks>
public abstract string ETag { get; }

/// <summary>
/// Gets the cosmos diagnostic information for the current request to Azure Cosmos DB service
/// </summary>
public CosmosDiagnostic cosmosDiagnostic { get; set; }

/// <summary>
/// Gets the maximum size limit for this entity from the Azure Cosmos DB service.
/// </summary>
Expand Down
33 changes: 33 additions & 0 deletions Microsoft.Azure.Cosmos/src/Resource/Settings/CosmosDiagnostic.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
//------------------------------------------------------------
// Copyright (c) Microsoft Corporation. All rights reserved.
//------------------------------------------------------------

namespace Microsoft.Azure.Cosmos
{
/// <summary>
/// Contains the cosmos diagnostic information for the current request to Azure Cosmos DB service.
/// </summary>
public class CosmosDiagnostic
simplynaveen20 marked this conversation as resolved.
Show resolved Hide resolved
{
internal PointOperationStatistics pointOperationStatistics { get; set; }
internal QueryOperationStatistics queryOperationStatistics { get; set; }

/// <summary>
/// Gets the string field <see cref="Microsoft.Azure.Cosmos.CosmosDiagnostic"/> instance in the Azure DocumentDB database service.
/// </summary>
/// <returns>The string field <see cref="Microsoft.Azure.Cosmos.CosmosDiagnostic"/> instance in the Azure DocumentDB database service.</returns>
public override string ToString()
{
if (pointOperationStatistics != null)
{
return pointOperationStatistics.ToString();
}
else if (queryOperationStatistics != null)
{
return queryOperationStatistics.ToString();
}

return string.Empty;
}
}
}
Loading