Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Monitor query updates #17459

Merged
merged 58 commits into from
Sep 9, 2021
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
b2c7bad
rename column to columnDescriptors and enums for error status
KarishmaGhiya Sep 3, 2021
97492a4
sample for second metric
KarishmaGhiya Sep 3, 2021
a00fe8c
update metrics query client
KarishmaGhiya Sep 3, 2021
57216a4
result status
KarishmaGhiya Sep 3, 2021
772a1ca
update samples
KarishmaGhiya Sep 3, 2021
a743827
getMetricByNames, flag for throwOnAnyFailure
KarishmaGhiya Sep 4, 2021
4f74a91
refdocs and samples
KarishmaGhiya Sep 4, 2021
206a91c
regenerate swagger with innerError rename
KarishmaGhiya Sep 4, 2021
6b5ab13
regenerate swagger with innerError rename
KarishmaGhiya Sep 4, 2021
26a3440
update api review
KarishmaGhiya Sep 4, 2021
a427687
fix lint errors
KarishmaGhiya Sep 7, 2021
ff7cdfd
update metricsQuery sample
KarishmaGhiya Sep 7, 2021
c75643f
update KQL for multiple workspaces
KarishmaGhiya Sep 7, 2021
7b9020d
update readme
KarishmaGhiya Sep 7, 2021
227bbb8
update readme
KarishmaGhiya Sep 7, 2021
60df0c5
update readme
KarishmaGhiya Sep 7, 2021
dbc6cd4
update readme
KarishmaGhiya Sep 7, 2021
0941991
Update sdk/monitor/monitor-query/README.md
KarishmaGhiya Sep 7, 2021
31987fd
Update sdk/monitor/monitor-query/README.md
KarishmaGhiya Sep 7, 2021
a929e08
Update sdk/monitor/monitor-query/README.md
KarishmaGhiya Sep 7, 2021
3a54e4d
Update sdk/monitor/monitor-query/README.md
KarishmaGhiya Sep 7, 2021
cbd309b
Update sdk/monitor/monitor-query/src/models/publicLogsModels.ts
KarishmaGhiya Sep 7, 2021
edf39ca
Update sdk/monitor/monitor-query/README.md
KarishmaGhiya Sep 7, 2021
b924d3d
Update sdk/monitor/monitor-query/README.md
KarishmaGhiya Sep 7, 2021
f64a7fc
Update sdk/monitor/monitor-query/README.md
KarishmaGhiya Sep 7, 2021
8636437
cleanup console.log statements
KarishmaGhiya Sep 7, 2021
5a21ffb
Merge branch 'monitor-query-updates' of https://github.com/KarishmaGh…
KarishmaGhiya Sep 7, 2021
5e6d08b
update optional param to be required
KarishmaGhiya Sep 7, 2021
7038eb4
rename variables
KarishmaGhiya Sep 7, 2021
fa238a8
Update sdk/monitor/monitor-query/src/models/publicLogsModels.ts
KarishmaGhiya Sep 7, 2021
41f82df
update api review file and samples
KarishmaGhiya Sep 8, 2021
2ff1efb
update sample
KarishmaGhiya Sep 8, 2021
5ac02ad
Merge branch 'monitor-query-updates' of https://github.com/KarishmaGh…
KarishmaGhiya Sep 8, 2021
a17c343
update readme
KarishmaGhiya Sep 8, 2021
6748d77
TimeInterval return type for timespan in MetricsQueryResult
KarishmaGhiya Sep 8, 2021
3bda4b4
fix tests
KarishmaGhiya Sep 8, 2021
5c930f2
update conversion
KarishmaGhiya Sep 8, 2021
fed387c
Update sdk/monitor/monitor-query/src/internal/modelConverters.ts
KarishmaGhiya Sep 8, 2021
a0cec0e
check if this works on ci
KarishmaGhiya Sep 8, 2021
9e6f633
check if this works on ci
KarishmaGhiya Sep 8, 2021
75c62d6
update conversion
KarishmaGhiya Sep 8, 2021
da9a5c1
update readme
KarishmaGhiya Sep 8, 2021
8a9ee1b
added changelog
KarishmaGhiya Sep 9, 2021
265e265
rename
KarishmaGhiya Sep 9, 2021
170dd4a
readme
KarishmaGhiya Sep 9, 2021
f89cd47
Update sdk/monitor/monitor-query/src/internal/modelConverters.ts
KarishmaGhiya Sep 9, 2021
d2ea6ce
update Error types for batch and updated samples
KarishmaGhiya Sep 9, 2021
51b3106
Merge branch 'monitor-query-updates' of https://github.com/KarishmaGh…
KarishmaGhiya Sep 9, 2021
6cdc3c0
update models
KarishmaGhiya Sep 9, 2021
aa05c8a
update models
KarishmaGhiya Sep 9, 2021
6adbbb8
update changelog
KarishmaGhiya Sep 9, 2021
d762918
update api review
KarishmaGhiya Sep 9, 2021
6554230
update reamde sample
KarishmaGhiya Sep 9, 2021
41ba681
Update sdk/monitor/monitor-query/CHANGELOG.md
KarishmaGhiya Sep 9, 2021
6479744
Update sdk/monitor/monitor-query/README.md
KarishmaGhiya Sep 9, 2021
74baa6f
Update sdk/monitor/monitor-query/README.md
KarishmaGhiya Sep 9, 2021
a5d0d52
update logic for single query status compute
KarishmaGhiya Sep 9, 2021
e78334e
Merge branch 'monitor-query-updates' of https://github.com/KarishmaGh…
KarishmaGhiya Sep 9, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion sdk/monitor/monitor-query/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@
"build:samples": "dev-tool samples publish --force",
"build:test": "tsc -p . && rollup -c 2>&1",
"build:types": "downlevel-dts types/latest types/3.1",
"build": "npm run clean && tsc -p . && npm run lint && npm run build:nodebrowser && api-extractor run --local && npm run build:types",
"build": "npm run clean && tsc -p . && npm run build:nodebrowser && api-extractor run --local && npm run build:types",
"check-format": "prettier --list-different --config ../../../.prettierrc.json --ignore-path ../../../.prettierignore \"src/**/*.ts\" \"test/**/*.ts\" \"*.{js,json}\"",
"clean": "rimraf dist dist-* temp types *.tgz *.log coverage coverage-browser",
"docs": "typedoc --excludePrivate --excludeNotExported --excludeExternals --stripInternal --mode file --out ./dist/docs ./src",
Expand Down
21 changes: 15 additions & 6 deletions sdk/monitor/monitor-query/review/monitor-query.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@
import { CommonClientOptions } from '@azure/core-client';
import { OperationOptions } from '@azure/core-client';
import { PagedAsyncIterableIterator } from '@azure/core-paging';
import { PipelineOptions } from '@azure/core-rest-pipeline';
import { TokenCredential } from '@azure/core-auth';

// @public
Expand Down Expand Up @@ -42,7 +41,7 @@ export interface ErrorInfo {
additionalProperties?: Record<string, unknown>;
code: string;
details?: ErrorDetail[];
innererror?: ErrorInfo;
innerError?: ErrorInfo;
message: string;
}

Expand All @@ -66,15 +65,19 @@ export interface LogsColumn {
export type LogsColumnType = string;

// @public
export type LogsQueryBatchOptions = OperationOptions;
export interface LogsQueryBatchOptions extends OperationOptions {
throwOnAnyFailure?: boolean;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we return a promise that can be rejected, I think rejectOnAnyFailure might be more accurate.

But remind me...so if I set this to true, then if any of the queries in the batch fails, the whole thing fails? The scenario you talked to me about before was disabling throwing an error on partial failures, so I'm curious about this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - this one will throw for both complete and partial failures. And the whole thing will fail in this case. But the users will get the data about all the responses. We wanted to follow similar approach across all JS sdks -

if (options.throwOnAnyFailure && result._response.status === 207) {
Search-documents uses that flag - so thought of using the same approach.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah gotcha, if search-documents already uses that flag, then disregard my suggestion. Rather keep it consistent with throwOnAnyFailure then.

}

// @public
export interface LogsQueryBatchResult {
// (undocumented)
batchResultStatus: "AllSucceeded" | "AllFailed" | "PartiallySucceeded";
KarishmaGhiya marked this conversation as resolved.
Show resolved Hide resolved
results?: {
id?: string;
status?: number;
tables?: LogsTable[];
error?: ErrorInfo;
logsQueryResultStatus?: LogsQueryResultStatus;
statistics?: Record<string, unknown>;
visualization?: Record<string, unknown>;
}[];
Expand All @@ -101,19 +104,24 @@ export interface LogsQueryOptions extends OperationOptions {
includeQueryStatistics?: boolean;
includeVisualization?: boolean;
serverTimeoutInSeconds?: number;
throwOnAnyFailure?: boolean;
}

// @public
export interface LogsQueryResult {
error?: ErrorInfo;
logsQueryResultStatus?: LogsQueryResultStatus;
KarishmaGhiya marked this conversation as resolved.
Show resolved Hide resolved
statistics?: Record<string, unknown>;
tables: LogsTable[];
visualization?: Record<string, unknown>;
}

// @public
export type LogsQueryResultStatus = "Partial" | "Success" | "Failed";
KarishmaGhiya marked this conversation as resolved.
Show resolved Hide resolved

// @public
export interface LogsTable {
columns: LogsColumn[];
columnDescriptors: LogsColumn[];
name: string;
rows: (Date | string | number | Record<string, unknown> | boolean)[][];
}
Expand Down Expand Up @@ -171,7 +179,7 @@ export interface MetricNamespace {
}

// @public
export interface MetricsClientOptions extends PipelineOptions {
export interface MetricsClientOptions extends CommonClientOptions {
endpoint?: string;
}

Expand Down Expand Up @@ -199,6 +207,7 @@ export interface MetricsQueryOptions extends OperationOptions {
// @public
export interface MetricsQueryResult {
cost?: number;
getMetricByName(metricName: string): Metric;
granularity?: string;
metrics: Metric[];
namespace?: string;
Expand Down
2 changes: 1 addition & 1 deletion sdk/monitor/monitor-query/samples-dev/logsQuery.ts
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ export async function main() {
);

for (const table of tablesFromResult) {
const columnHeaderString = table.columns
const columnHeaderString = table.columnDescriptors
.map((column) => `${column.name}(${column.type}) `)
.join("| ");
console.log("| " + columnHeaderString);
Expand Down
4 changes: 2 additions & 2 deletions sdk/monitor/monitor-query/samples-dev/logsQueryBatch.ts
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ export async function main() {

let i = 0;
for (const response of result.results) {
console.log(`Results for query with id: ${response.id}`);
console.log(`Results for query with query: ${queriesBatch[i]}`);

if (response.error) {
console.log(` Query had errors:`, response.error);
Expand All @@ -67,7 +67,7 @@ export async function main() {
);

for (const table of response.tables) {
const columnHeaderString = table.columns
const columnHeaderString = table.columnDescriptors
.map((column) => `${column.name}(${column.type}) `)
.join("| ");
console.log(columnHeaderString);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ export async function main() {
);

for (const table of tablesFromResult) {
const columnHeaderString = table.columns
const columnHeaderString = table.columnDescriptors
.map((column) => `${column.name}(${column.type}) `)
.join("| ");
console.log("| " + columnHeaderString);
Expand Down
17 changes: 12 additions & 5 deletions sdk/monitor/monitor-query/samples-dev/metricsQuery.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,26 +24,33 @@ export async function main() {
const iterator = metricsQueryClient.listMetricDefinitions(metricsResourceId);
let result = await iterator.next();
const firstMetric: MetricDefinition = result.value;

let secondMetricName: string;
while (!result.done) {
console.log(` metricDefinitions - ${result.value.id}, ${result.value.name}`);
secondMetricName = result.value.name!;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use for await(const result of iterator) instead?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also...this has the potential to return lots of results right? Otherwise you'd exit the loop as soon as you got the 2nd metric. Can you collect all the names to pass to the query method?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah we can collect all the names and pass it to query method. Should I change the sample to that?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am thinking of letting this be - and may be adding a comment that they can pass in all the names to the query method. What do you think? @chradek

result = await iterator.next();
}
console.log(`First Metric Definition = ${firstMetric.name}`);

console.log(`Picking an example metric to query: ${firstMetric.name!}`);

const metricsResponse = await metricsQueryClient.query(metricsResourceId, [firstMetric.name!], {
granularity: "PT1M",
timespan: { duration: Durations.FiveMinutes }
});
const metricsResponse = await metricsQueryClient.query(
metricsResourceId,
[firstMetric.name!, secondMetricName!],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can name really be undefined? Is that just because of the swagger model, or can the service really return an undefined name? Using the non-null assertion always makes me suspicious.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bcz of swagger model - Metric Definitions and Metric Namespaces can return empty objects as specified in the swagger by the service.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get that the swagger model models the name as optional, but my question is, does the service actually treat name as optional or is it a mistake in the model? We're using the non-null assertion in our sample, which either means we believe it is actually always defined - so why make it optional - or we're not showing the customer how to handle a legitimate edge case.

If name was actually undefined, would that be an error? What happens if you end up passing undefined to query?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we pass undefined to query the service will return an error - so we shouldn't do that. I updated the sample to do the verification of both metric names being valid

{
granularity: "PT1M",
timespan: { duration: Durations.FiveMinutes }
}
);

console.log(
`Query cost: ${metricsResponse.cost}, interval: ${metricsResponse.granularity}, time span: ${metricsResponse.timespan}`
);

const metrics: Metric[] = metricsResponse.metrics;
console.log(`Metrics:`, JSON.stringify(metrics, undefined, 2));
const metric = metricsResponse.getMetricByName(firstMetric.name!);
console.log(`Selected Metric: ${firstMetric.name}`, JSON.stringify(metric, undefined, 2));
}

main().catch((err) => {
Expand Down
10 changes: 5 additions & 5 deletions sdk/monitor/monitor-query/samples/v1/javascript/logsQuery.js
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ async function main() {
const kustoQuery =
"AppEvents | project TimeGenerated, Name, AppRoleInstance | order by TimeGenerated asc | limit 10";

console.log(`Running '${kustoQuery}' over the last 5 minutes`);
console.log(`Running '${kustoQuery}' over the last One Hour`);
const queryLogsOptions = {
// explicitly control the amount of time the server can spend processing the query.
serverTimeoutInSeconds: 60,
Expand All @@ -32,13 +32,13 @@ async function main() {
includeQueryStatistics: true
};

const result = await logsQueryClient.queryLogs(
const result = await logsQueryClient.query(
monitorWorkspaceId,
kustoQuery,
// The timespan is an ISO8601 formatted time (or interval). Some common aliases
// are available (like lastDay, lastHour, last48Hours, etc..) but any properly formatted ISO8601
// are available (like OneDay, OneHour, FoutyEightHours, etc..) but any properly formatted ISO8601
// value is valid.
Durations.lastHour,
{ duration: Durations.OneHour },
KarishmaGhiya marked this conversation as resolved.
Show resolved Hide resolved
queryLogsOptions
);

Expand All @@ -59,7 +59,7 @@ async function main() {
);

for (const table of tablesFromResult) {
const columnHeaderString = table.columns
const columnHeaderString = table.columnDescriptors
.map((column) => `${column.name}(${column.type}) `)
.join("| ");
console.log("| " + columnHeaderString);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,38 +25,36 @@ async function main() {
{
workspaceId: monitorWorkspaceId,
query: kqlQuery,
timespan: "P1D"
timespan: { duration: "P1D" }
},
{
workspaceId: monitorWorkspaceId,
query: "AzureActivity | summarize count()",
timespan: "PT1H"
timespan: { duration: "PT1H" }
},
{
workspaceId: monitorWorkspaceId,
query:
"AppRequests | take 10 | summarize avgRequestDuration=avg(DurationMs) by bin(TimeGenerated, 10m), _ResourceId",
timespan: "PT1H"
timespan: { duration: "PT1H" }
},
{
workspaceId: monitorWorkspaceId,
query: "AppRequests | take 2",
timespan: "PT1H",
timespan: { duration: "PT1H" },
includeQueryStatistics: true
}
];

const result = await logsQueryClient.queryLogsBatch({
queries: queriesBatch
});
const result = await logsQueryClient.queryBatch(queriesBatch);

if (result.results == null) {
throw new Error("No response for query");
}

let i = 0;
for (const response of result.results) {
console.log(`Results for query with id: ${response.id}`);
console.log(`Results for query with query: ${queriesBatch[i]}`);

if (response.error) {
console.log(` Query had errors:`, response.error);
Expand All @@ -69,7 +67,7 @@ async function main() {
);

for (const table of response.tables) {
const columnHeaderString = table.columns
const columnHeaderString = table.columnDescriptors
.map((column) => `${column.name}(${column.type}) `)
.join("| ");
console.log(columnHeaderString);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,13 +35,13 @@ async function main() {
additionalWorkspaces: [additionalWorkspaces1, additionalWorkspaces2]
};

const result = await logsQueryClient.queryLogs(
const result = await logsQueryClient.query(
monitorWorkspaceId,
kustoQuery,
// The timespan is an ISO8601 formatted time (or interval). Some common aliases
// are available (like lastDay, lastHour, last48Hours, etc..) but any properly formatted ISO8601
// are available (like durationOf1Day, durationOf1Hour, durationOf48Hours, etc..) but any properly formatted ISO8601
// value is valid.
Durations.lastHour,
{ duration: Durations.OneHour },
queryLogsOptions
);

Expand All @@ -62,7 +62,7 @@ async function main() {
);

for (const table of tablesFromResult) {
const columnHeaderString = table.columns
const columnHeaderString = table.columnDescriptors
.map((column) => `${column.name}(${column.type}) `)
.join("| ");
console.log("| " + columnHeaderString);
Expand Down
27 changes: 16 additions & 11 deletions sdk/monitor/monitor-query/samples/v1/javascript/metricsQuery.js
Original file line number Diff line number Diff line change
Expand Up @@ -21,31 +21,36 @@ async function main() {
throw new Error("METRICS_RESOURCE_ID must be set in the environment for this sample");
}

const result = await metricsQueryClient.getMetricDefinitions(metricsResourceId);

for (const definition of result.definitions) {
console.log(`Definition = ${definition.name}`);
const iterator = metricsQueryClient.listMetricDefinitions(metricsResourceId);
let result = await iterator.next();
const firstMetric = result.value;
let secondMetricName;
while (!result.done) {
console.log(` metricDefinitions - ${result.value.id}, ${result.value.name}`);
secondMetricName = result.value.name;
result = await iterator.next();
}

const firstMetric = result.definitions[0];
console.log(`First Metric Definition = ${firstMetric.name}`);

console.log(`Picking an example metric to query: ${firstMetric.name}`);

const metricsResponse = await metricsQueryClient.queryMetrics(
const metricsResponse = await metricsQueryClient.query(
metricsResourceId,
Durations.last5Minutes,
[firstMetric.name, secondMetricName],
KarishmaGhiya marked this conversation as resolved.
Show resolved Hide resolved
{
metricNames: [firstMetric.name],
interval: "PT1M"
granularity: "PT1M",
timespan: { duration: Durations.FiveMinutes }
}
);

console.log(
`Query cost: ${metricsResponse.cost}, interval: ${metricsResponse.interval}, time span: ${metricsResponse.timespan}`
`Query cost: ${metricsResponse.cost}, interval: ${metricsResponse.granularity}, time span: ${metricsResponse.timespan}`
);

const metrics = metricsResponse.metrics;
console.log(`Metrics:`, JSON.stringify(metrics, undefined, 2));
const metric = metricsResponse.getMetricByName(firstMetric.name);
console.log(`Selected Metric: ${firstMetric.name}`, JSON.stringify(metric, undefined, 2));
}

main().catch((err) => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
*/

import { DefaultAzureCredential } from "@azure/identity";
import { Durations, LogsQueryClient, LogsTable, QueryLogsOptions } from "@azure/monitor-query";
import { Durations, LogsQueryClient, LogsTable, LogsQueryOptions } from "@azure/monitor-query";
import * as dotenv from "dotenv";
dotenv.config();

Expand All @@ -23,22 +23,22 @@ export async function main() {
const kustoQuery =
"AppEvents | project TimeGenerated, Name, AppRoleInstance | order by TimeGenerated asc | limit 10";

console.log(`Running '${kustoQuery}' over the last 5 minutes`);
const queryLogsOptions: QueryLogsOptions = {
console.log(`Running '${kustoQuery}' over the last One Hour`);
const queryLogsOptions: LogsQueryOptions = {
// explicitly control the amount of time the server can spend processing the query.
serverTimeoutInSeconds: 60,
// optionally enable returning additional statistics about the query's execution.
// (by default this is off)
includeQueryStatistics: true
};

const result = await logsQueryClient.queryLogs(
const result = await logsQueryClient.query(
monitorWorkspaceId,
kustoQuery,
// The timespan is an ISO8601 formatted time (or interval). Some common aliases
// are available (like lastDay, lastHour, last48Hours, etc..) but any properly formatted ISO8601
// are available (like OneDay, OneHour, FoutyEightHours, etc..) but any properly formatted ISO8601
// value is valid.
Durations.lastHour,
{ duration: Durations.OneHour },
queryLogsOptions
);

Expand All @@ -50,7 +50,7 @@ export async function main() {
}

const executionTime =
result.statistics && result.statistics.query && result.statistics.query.executionTime;
result.statistics && result.statistics.query && (result.statistics.query as any).executionTime;

console.log(
`Results for query '${kustoQuery}', execution time: ${
Expand All @@ -59,7 +59,7 @@ export async function main() {
);

for (const table of tablesFromResult) {
const columnHeaderString = table.columns
const columnHeaderString = table.columnDescriptors
.map((column) => `${column.name}(${column.type}) `)
.join("| ");
console.log("| " + columnHeaderString);
Expand Down
Loading