Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(cloudwatch): dashboard variables #26285

Merged
merged 27 commits into from
Jul 11, 2023
Merged

feat(cloudwatch): dashboard variables #26285

merged 27 commits into from
Jul 11, 2023

Conversation

Stacy-D
Copy link
Contributor

@Stacy-D Stacy-D commented Jul 7, 2023

This change add the support for dashboard variables in CDK https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_dashboard_variables.html. It allows to reduce the number of repeated CloudWatch dashboards by unifying them into one managed with variables.

Closes #26200


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

- Add dashboard variables implementation
- Add tests
- Add integ test
- Add section in README on variables
@gitpod-io
Copy link

gitpod-io bot commented Jul 7, 2023

@github-actions github-actions bot added repeat-contributor [Pilot] contributed between 3-5 PRs to the CDK effort/medium Medium work item – several days of effort feature-request A feature should be added or improved. p2 labels Jul 7, 2023
@aws-cdk-automation aws-cdk-automation requested a review from a team July 7, 2023 15:53
@Stacy-D
Copy link
Contributor Author

Stacy-D commented Jul 7, 2023

Example dashboard with a search variable created as a result of added integ test:

Screenshot 2023-07-07 at 17 02 02

@aws-cdk-automation aws-cdk-automation added the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Jul 7, 2023
@humanzz
Copy link
Contributor

humanzz commented Jul 7, 2023

Hello reviewer,
Ana and I had reached out to the cloudwatch team to update the documentation describing the dashboard json to include dashboard variables to help you with reviewing these changes. The documentation update had just been released at https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/CloudWatch-Dashboard-Body-Structure.html#CloudWatch-Dashboard-Properties-Variables-Structure

Copy link
Contributor

@kaizencc kaizencc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Stacy-D. Thanks for this PR -- it is a great start! Just a few nits here and there and sorry that I did not like the phrase allow to :)

packages/aws-cdk-lib/aws-cloudwatch/README.md Outdated Show resolved Hide resolved
packages/aws-cdk-lib/aws-cloudwatch/README.md Outdated Show resolved Hide resolved
packages/aws-cdk-lib/aws-cloudwatch/README.md Outdated Show resolved Hide resolved
packages/aws-cdk-lib/aws-cloudwatch/README.md Outdated Show resolved Hide resolved
packages/aws-cdk-lib/aws-cloudwatch/README.md Outdated Show resolved Hide resolved
packages/aws-cdk-lib/aws-cloudwatch/lib/variable.ts Outdated Show resolved Hide resolved
packages/aws-cdk-lib/aws-cloudwatch/lib/variable.ts Outdated Show resolved Hide resolved
packages/aws-cdk-lib/aws-cloudwatch/test/dashboard.test.ts Outdated Show resolved Hide resolved
packages/aws-cdk-lib/aws-cloudwatch/lib/dashboard.ts Outdated Show resolved Hide resolved
@kaizencc kaizencc self-assigned this Jul 7, 2023
@humanzz
Copy link
Contributor

humanzz commented Jul 8, 2023

Hey @kaizencc
It's great that this overall looks good to you!
@Stacy-D is taking this week off and is likely unable to address things during that time. I'll try to address them in her absence so expect changes from me soon :)

Co-authored-by: Kaizen Conroy <36202692+kaizencc@users.noreply.github.com>
@mergify mergify bot dismissed kaizencc’s stale review July 8, 2023 09:13

Pull request has been modified.

@Stacy-D
Copy link
Contributor Author

Stacy-D commented Jul 8, 2023

Updated result of the integration test with both search and value variables

Screenshot 2023-07-08 at 11 40 58

@Stacy-D
Copy link
Contributor Author

Stacy-D commented Jul 8, 2023

Hey @kaizencc,
Thanks for your review! I addressed the requested changes and responded to your questions. So I'm now looking forward to a second review 😄

All the follow ups, should those be needed, will be addressed by @humanzz while I'm off.

@humanzz
Copy link
Contributor

humanzz commented Jul 8, 2023

@kaizencc just a question, re the enums introduced in this PR, in previous PRs we sometimes got the feedback to prefer classes with static members to kinda future proof against introductions of new types e.g. https://github.com/aws/aws-cdk/blob/main/packages/aws-cdk-lib/aws-lambda/lib/architecture.ts.
Any thoughts on this?

Copy link
Contributor

@kaizencc kaizencc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pretty much looks good to go to me @Stacy-D @humanzz! Just a few more minor comments.

packages/aws-cdk-lib/aws-cloudwatch/lib/variable.ts Outdated Show resolved Hide resolved
packages/aws-cdk-lib/aws-cloudwatch/lib/variable.ts Outdated Show resolved Hide resolved
export class SearchDashboardVariable extends DashboardVariable {
private readonly props: ISearchDashboardVariable;

constructor(props: ISearchDashboardVariable) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the API docs, search and populateFrom are required "if inputType is select or radio and you are not specifying values."

Just want to confirm that SearchDashboardVariable will still work with inputType input right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, I went to try a different combo of parameters manually, and what would essentially happen if input inputType and we have a search variable, is that it will just be a normal input. However, on saving one is faced with the nice error

The dashboard body is invalid, there are 2 validation errors: [ { "dataPath": "/variables/0/search", "message": "Should NOT be valid" }, { "dataPath": "/variables/0/populateFrom", "message": "Should NOT be valid" } ]

So, to me there are 2 things to do here

  1. SearchDashboardVariable should validate that inputType is not input
  2. ISearchDashboardVariable.populateFrom should not be optional

I think ValueDashboardVariable might need some similar validations as well

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lovely. Thanks for the investigation. That's what I was afraid of, but the docs don't really say it...

packages/aws-cdk-lib/aws-cloudwatch/lib/variable.ts Outdated Show resolved Hide resolved
/**
* Optional label for the selected item
*
* @default - value
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does value mean here? It's kind of an oversaturated word... if you mean the value you specify in the 'value' prop, then lets be specific :)

*/
readonly searchExpression: string;
/**
* Optional dimension name from the search
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

needs a @default. Not sure why this wouldn't throw an error somewhere in our PR build. Will have to take a look later.

packages/aws-cdk-lib/aws-cloudwatch/lib/variable.ts Outdated Show resolved Hide resolved
packages/aws-cdk-lib/aws-cloudwatch/lib/variable.ts Outdated Show resolved Hide resolved
@mergify mergify bot dismissed kaizencc’s stale review July 10, 2023 22:33

Pull request has been modified.

@kaizencc kaizencc changed the title feat(cloudwatch): introduce dashboard variables feat(cloudwatch): dashboard variables Jul 10, 2023
@humanzz
Copy link
Contributor

humanzz commented Jul 11, 2023

@kaizencc made a change at 6e6bdb5 and 4213942 where

  • I addressed the comments
  • made some renames
  • added some validations + tests

Hopefully this will make things a bit clearer for the users.

Copy link
Contributor

@kaizencc kaizencc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@humanzz Looks awesome! 3 last minute comments. Otherwise I'm happy with this :).


// WHEN
dashboard.addVariable(new SearchDashboardVariable({
defaultValue: '__FIRST',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just realized that __FIRST is a special value. Can we make that into a constant so that it can be referenced like:

defaultValue: `DefaultValue.FIRST`

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so just a class with a static constant?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WDYT? Maybe its unnecessary.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, I made it take a bit more responsibility in d8cc14a, let me know what you think

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like it. Will make DefaultValue.FIRST more discoverable, and I expect that it will be a popular default value for most use cases.

*
* For example `{AWS/EC2,InstanceId} MetricName=\"CPUUtilization\"`
*/
readonly searchExpression: string;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you given any thought to strongly typing this expression? How unique can this string be? The examples in documentation all reference a namespace, dimension, and metricName in some format. Maybe with stronger types we can build this expression out for the user and not have to have them worry about typos.

I'm thinking something like:

class searchExpression {
  public static FROM_COMPONENTS(
    namespace: string,
    dimension: string,
    metricName: string,
  ) {
    return new searchExpression(`{${namespace},${dimension}} MetricName=\"${metricName\"`);
  }

  // users can still supply raw expressions if they want
  public constructor(public readonly expression: string) {}
}

WDYT?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, I'd be a bit hesitant here, without further investigation, abt how metrics with multiple dimensions work - if they're supported or not.
If you think it's needed, I can give that a try and see if that's supported, in which case the search expression would need to allow for number of dimensions >= 1

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

multiple dimensions are supported... I'll work to create a class for this

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! I think if possible, a class would be a better customer experience than a raw string.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I blame you for the more code - now it needs more reviewing 😆

d8cc14a

I ended up refactoring how search values are modeled!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In fact, with this new change, I'm now wondering if we really need a separate ValueDashboardVariable and SearchDashboardVariable. The difference can really be just about the values prop. Something like Values.fromSearch and Values.fromStatic (or any better name).

Worth pursuing?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went ahead and did that. I believe it looks better.
Let me know what you think.
I also believe docs/README/error messages might need a good look to ensure they make sense for users.

packages/aws-cdk-lib/aws-cloudwatch/README.md Outdated Show resolved Hide resolved
@mergify mergify bot dismissed kaizencc’s stale review July 11, 2023 14:32

Pull request has been modified.

Copy link
Contributor

@kaizencc kaizencc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay! I think this is the last review. We're almost there. Thanks for bearing with me @humanzz


// WHEN
dashboard.addVariable(new SearchDashboardVariable({
defaultValue: '__FIRST',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like it. Will make DefaultValue.FIRST more discoverable, and I expect that it will be a popular default value for most use cases.

* @param metricName the metric name to be used in the search expression
* @param populateFrom the dimension name, that the search expression retrieves, whose values will be used to populate the values to choose from
*/
public static fromSearchComponents(namespace: string, dimensions: string[], metricName: string, populateFrom: string): Values {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

export interface SearchComponentProps {
  readonly namespace: string,
  readonly dimensions: string[],
  readonly metricName: string,
  readonly populateFrom: string,
}

export class Values {
  public static fromSearchComponents(props: SearchComponentProps): Values {
  }
}

packages/aws-cdk-lib/aws-cloudwatch/lib/variable.ts Outdated Show resolved Hide resolved
label: 'Function',
inputType: cw.VariableInputType.RADIO,
value: 'originalFuncNameInDashboard',
values: cw.Values.fromSearchComponents('AWS/Lambda', ['FunctionName'], 'Duration', 'FunctionName'), // equivalent to cw.Values.fromSearch('{AWS/Lambda,FunctionName} MetricName=\"Duration\"', 'FunctionName'),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
values: cw.Values.fromSearchComponents('AWS/Lambda', ['FunctionName'], 'Duration', 'FunctionName'), // equivalent to cw.Values.fromSearch('{AWS/Lambda,FunctionName} MetricName=\"Duration\"', 'FunctionName'),
// equivalent to cw.Values.fromSearch('{AWS/Lambda,FunctionName} MetricName=\"Duration\"', 'FunctionName'),
values: cw.Values.fromSearchComponents('AWS/Lambda', ['FunctionName'], 'Duration', 'FunctionName'),

packages/aws-cdk-lib/aws-cloudwatch/lib/variable.ts Outdated Show resolved Hide resolved
packages/aws-cdk-lib/aws-cloudwatch/lib/variable.ts Outdated Show resolved Hide resolved
throw new Error(`Variable with inputType (${options.inputType}) requires values to be set`);
}
if (options.inputType == VariableInputType.INPUT && options.values) {
throw new Error('Unsupported inputType INPUT. Please choose either SELECT or RADIO');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
throw new Error('Unsupported inputType INPUT. Please choose either SELECT or RADIO');
throw new Error('inputType INPUT cannot be combined with values. Please choose either SELECT or RADIO or remove \'values\' from the configuration.');


The following example generates a Lambda function variable, with a radio button for each function. Functions are discovered by a metric query search.
The `values` with `cw.Values.fromSearchComponents` indicates that the values will be populated from `FunctionName` values retrieved from the search expression `{AWS/Lambda,FunctionName} MetricName=\"Duration\"`.
The `defaultValue` with `cw.DefaultValue.FIRST` indicates that the default value will be the first value returned from the search.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The `defaultValue` with `cw.DefaultValue.FIRST` indicates that the default value will be the first value returned from the search.
The `defaultValue` with `cw.DefaultValue.FIRST` indicates that the default value will be the first value returned from the search.

Comment on lines 141 to 147
/**
* Create a default value
* @param value the value to be used as default
*/
public static of(value: any) {
return new DefaultValue(value);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't love of. Reason being that we're not creating a default value 'of' something. We're just referencing a value. Some options:

DefaultValue.string('us-east-1')
DefaultValue.value('us-east-1') // contributes to overkill of the word value though

I don't really care. I just dont think of is right.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I honestly can't think of a better option than of - except maybe the overkill value you suggested :)
string doesn't align well with the fact that it's actually any, with the documentation saying Type: String, Number, or Boolean, depending on the type value for this variable

I think I'll go with value

@mergify mergify bot dismissed kaizencc’s stale review July 11, 2023 19:30

Pull request has been modified.

@humanzz
Copy link
Contributor

humanzz commented Jul 11, 2023

Okay! I think this is the last review. We're almost there. Thanks for bearing with me @humanzz

no worries at all. I think we're making good progress here.

Copy link
Contributor

@kaizencc kaizencc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sweet @humanzz! How wonderful to get multiple rounds of review bashed out in a day :). Really appreciate the fast turnaround time!

@humanzz
Copy link
Contributor

humanzz commented Jul 11, 2023

Awesome @kaizencc!
Thanks for the quick and actionable reviews!
Let's hope we catch the next version train 🚄 ✨

@mergify
Copy link
Contributor

mergify bot commented Jul 11, 2023

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: 7912c6d
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@mergify mergify bot merged commit 73f2741 into aws:main Jul 11, 2023
@mergify
Copy link
Contributor

mergify bot commented Jul 11, 2023

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

bmoffatt pushed a commit to bmoffatt/aws-cdk that referenced this pull request Jul 29, 2023
This change add the support for dashboard variables in CDK https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_dashboard_variables.html. It allows to reduce the number of repeated  CloudWatch dashboards by unifying them into one managed with variables.


Closes aws#26200

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
effort/medium Medium work item – several days of effort feature-request A feature should be added or improved. p2 pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. repeat-contributor [Pilot] contributed between 3-5 PRs to the CDK
Projects
None yet
Development

Successfully merging this pull request may close these issues.

cloudwatch: support dashboard variables
4 participants