"Scanning CloudFormation templates..." notification never goes away/finishes #3510
Labels
bug
We can reproduce the issue and confirmed it is a bug.
pending-release
performance
service:cloudformation
Comments
ConnorChristie
added
the
bug
Disabling the We are working on more total performance improvements: #3216 |
Thank you but it seems that option was already disabled for me. Tried toggling it and that did not make a difference. |
|
Adding to this, it seems that whenever the scan is in progress, all other extensions stop working. This is making it impossible for me to continue using the extension, as even with |
@GPhilo can you try vscode's
|
|
I'll enable the setting and give it an in-depth try tomorrow, from a first look it seems that it improves the situation greatly already. |
|
So, after a few days with the affinity setting enabled, I still notice very big delays in other extensions while the "Scanning CloudFormation templates..." task is running. |
Not yet, but performance is one of our priorities in the near/medium-term. |
justinmk3
added a commit
that referenced
this issue
Aug 24, 2023
Problem: "Scanning CloudFormation templates..." takes a long time on big workspaces, and there is no workaround. #3510 Solution: When scanning for CFN/SAM templates, exclude all directories/files specified in _all_ of the vscode settings `files.exclude`, `search.exclude`, or `files.watcherExclude`.
justinmk3
added a commit
that referenced
this issue
Aug 24, 2023
Problem: "Scanning CloudFormation templates..." takes a long time on big workspaces, and there is no workaround. #3510 Solution: When scanning for CFN/SAM templates, exclude all directories/files specified in _all_ of the vscode settings `files.exclude`, `search.exclude`, or `files.watcherExclude`.
justinmk3
added a commit
that referenced
this issue
Aug 24, 2023
Problem: "Scanning CloudFormation templates..." takes a long time on big workspaces, and there is no workaround. #3510 Solution: When scanning for CFN/SAM templates, exclude all directories/files specified in _all_ of the vscode settings `files.exclude`, `search.exclude`, or `files.watcherExclude`.
justinmk3
added a commit
that referenced
this issue
Aug 24, 2023
Problem: "Scanning CloudFormation templates..." takes a long time on big workspaces, and there is no workaround. #3510 Solution: When scanning for CFN/SAM templates, exclude all directories/files specified in _all_ of the vscode settings `files.exclude`, `search.exclude`, or `files.watcherExclude`.
justinmk3
added a commit
that referenced
this issue
Aug 24, 2023
Problem: "Scanning CloudFormation templates..." takes a long time on big workspaces, and there is no workaround. #3510 Solution: When scanning for CFN/SAM templates, exclude all directories/files specified in _all_ of the vscode settings `files.exclude`, `search.exclude`, or `files.watcherExclude`.
Implemented in AWS Toolkit 1.86. All directories mentioned by the standard vscode settings That provides a flexible workaround, but the root cause of the performance issue could still be improved, so leaving this issue open. |
|
I think it would at least be good to add one non-standard-vscode setting to that list, so that one could use that setting to exclude parts of hiw workspace, while not affecting other VSCode functionalities. |
This is not helpful. The CloudFormation scan hangs vscode at startup. I don't want the scan at all, but I want search to work for my other extensions and find in files and the global setting disables that. |
|
Supporting "exclude patterns" was a low-risk way to ameliorate some (but not all) scenarios. The next step in improving this behavior is to make it "just in time" so that it doesn't try to scan until SAM features are actually requested. |
|
The correct approach is to allow users who do not have CloudFormations to disable it. Not exclude locations. ZERO scanning. |
|
@justinmk3 Do we have a timeline on when (or if) this setting (to disable scans) be released? |
|
Hey all, I'm taking a look into this right now...my suspicion isn't necessarily the file watcher's performance, but the fact that we're essentially loading every since YAML file in the workspace. Out of curiosity, do the repos that you're having problems with have a lot of |
|
@bryceitoc9 Thanks for taking a look into it. In my case, there is not a single .yaml/.yml files in my workspace. I appreciate us looking into it from the performance standpoint as we should but can we just skip this and have a separate setting to disable it altogether? We can continue looking into the performance issues for the people interested in these scans in parallel but having the setting will immediately benefits a lot of the users right now. |
|
Adding a setting to disable this is non-trivial because SAM features depend on it. The work needed to make a "disable" setting work is similar to the work needed for the scan to be "on-demand". Meanwhile you can disable this by setting |
|
Unfortunately, disabling via |
|
Understood. This issue is still open and we're continuing to work on it. |
|
I'm not 100% sure what the sequencing is, but I've also merged #3931 , which was rescanning the workspace multiple times for each runtime we support...I'm not sure if VS Code attempts to run this scanner at the same time as the YAML one (where the message is generated from). If you want to give this a crack before the next release, you can try the latest prerelease VSIX: |
justinmk3
added a commit
that referenced
this issue
Oct 26, 2023
Problem: The AWS Documents handler (`awsFiletypes.ts`) is triggered frequently. It calls `globals.templateRegistry` which triggers an expensive "Scanning CloudFormation templates..." process. #3510 Solution: Extract the validation logic out of `CloudFormationTemplateRegistry.process()` so that `awsFiletypes.ts` can use it without requesting the full registry.
justinmk3
added a commit
that referenced
this issue
Oct 26, 2023
Problem: The AWS Documents handler (`awsFiletypes.ts`) is triggered frequently. It calls `globals.templateRegistry` which triggers an expensive "Scanning CloudFormation templates..." process. #3510 Solution: Extract the validation logic out of `CloudFormationTemplateRegistry.process()` so that `awsFiletypes.ts` can use it without requesting the full registry.
justinmk3
added a commit
that referenced
this issue
Oct 26, 2023
Problem: The AWS Documents handler (`awsFiletypes.ts`) is triggered frequently. It calls `globals.templateRegistry` which triggers an expensive "Scanning CloudFormation templates..." process. #3510 Solution: Extract the validation logic out of `CloudFormationTemplateRegistry.process()` so that `awsFiletypes.ts` can use it without requesting the full registry.
justinmk3
added a commit
that referenced
this issue
Oct 26, 2023
Problem: The AWS Documents handler (`awsFiletypes.ts`) is triggered frequently. It calls `globals.templateRegistry` which triggers an expensive "Scanning CloudFormation templates..." process. #3510 Solution: Extract the validation logic out of `CloudFormationTemplateRegistry.process()` so that `awsFiletypes.ts` can use it without requesting the full registry.
justinmk3
added a commit
that referenced
this issue
Oct 26, 2023
Problem: The AWS Documents handler (`awsFiletypes.ts`) is triggered frequently. It calls `globals.templateRegistry` which triggers an expensive "Scanning CloudFormation templates..." process. #3510 Solution: Extract the validation logic out of `CloudFormationTemplateRegistry.process()` so that `awsFiletypes.ts` can use it without requesting the full registry.
justinmk3
added a commit
that referenced
this issue
Oct 26, 2023
Problem: The AWS Documents handler (`awsFiletypes.ts`) is triggered frequently. It calls `globals.templateRegistry` which triggers an expensive "Scanning CloudFormation templates..." process. #3510 Solution: Extract the validation logic out of `CloudFormationTemplateRegistry.process()` so that `awsFiletypes.ts` can use it without requesting the full registry.
justinmk3
added a commit
that referenced
this issue
Oct 26, 2023
Problem: The AWS Documents handler (`awsFiletypes.ts`) is triggered frequently. It calls `globals.templateRegistry` which triggers an expensive "Scanning CloudFormation templates..." process. #3510 Solution: Extract the validation logic out of `CloudFormationTemplateRegistry.process()` so that `awsFiletypes.ts` can use it without requesting the full registry.
justinmk3
added a commit
that referenced
this issue
Oct 26, 2023
Problem: The AWS Documents handler (`awsFiletypes.ts`) is triggered frequently. It calls `globals.templateRegistry` which triggers an expensive "Scanning CloudFormation templates..." process. #3510 Solution: Extract the validation logic out of `CloudFormationTemplateRegistry.process()` so that `awsFiletypes.ts` can use it without requesting the full registry.
|
In AWS Toolkit 1.96, the "Scanning CloudFormation templates..." should no longer show except when using a SAM feature of AWS Toolkit. Currently it will still trigger if you open a SAM |
ctlai95
added a commit
that referenced
this issue
Oct 26, 2023
) * auth: ellipsis + right click options for `Developer Tools` (#3925) This commit allows user to right click to root CW or CC Node or click the new ellipsis icon, and they will get the options to Add Connection or Signout (if appropriate)   ## License By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. --------- Signed-off-by: nkomonen <nkomonen@amazon.com> * fix(codewihsperer): mapping language code before calling sendTelemetry API (#3928) * fix(codewhisperer): no auto trigger in Cloud9 #3927 * feat(settings): check that user settings.json is valid #3924 Problem: vscode notifies the user if `settings.json` is complete nonsense, but is silent if there are only "recoverable" JSON syntax errors. So _reading_ settings will appear to work, but _writing_ settings will throw an exception. #3910 Solution: On startup, check that we can actually _write_ settings, and show an error message if not. * watchedfiles: add watch patterns once to avoid rebuilding n times (#3931) Rrebuilding after each additional watch pattern meant we were processing globs n times, with each run containing `{1, 2, 3, ... , n-1, n}` patterns. Swapping this to an array and enforcing adding the patterns once ensures we only process 1 time, for n globs on initial load. We aren't adding globs dynamically at any point; if we ever feel the need to, we can implement some fancy caching/expiry business. FWIW it seems like getting the files in the glob is fairly fast, so if processing files is the bottleneck, this might be the cure? I'm sure we can improve more going forward regardless. <!--- REMINDER: - Read CONTRIBUTING.md first. - Add test coverage for your changes. - Update the changelog using `npm run newChange`. - Link to related issues/commits. - Testing: how did you test your changes? - Screenshots --> ## License By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. --------- Co-authored-by: Justin M. Keyes <jmkeyes@amazon.com> * test: skip e2e tests #3935 Reverts 7f98260 #3827 e2e tests are still always failing because: - webdriver html target is outdated - we are constantly throttled * build(deps): Bump the github-actions group with 1 update (#3937) Bumps the github-actions group with 1 update: [actions/setup-node](https://github.com/actions/setup-node). - [Release notes](https://github.com/actions/setup-node/releases) - [Commits](actions/setup-node@v3...v4) --- updated-dependencies: - dependency-name: actions/setup-node dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix(redshift): stale cache in connection wizard #3926 Problem: 1. When the connection parameters is removed, the re-launched connection wizard still has the earlier connection parameters. 2. When a sql query is successful but doesn't return any record, the output message is not clear. Solution: 1. Remove the connection wizard from the redshiftWarehouseNode constructor. When a connection wizard is launched, a new instance is created. 2. Modify the output message when a sql query is successful but doesn't return any record. * fix: Toolkit always opens Output panel on startup #3939 Problem: Output panel always opens at VS Code launch. #3930 Solution: Instead of showing Output panel at Redshift activation, change to show Output panel only when a message is added. * auth: Helper methods to check if connection exists (#3747) * auth: funcs to indicate if the connection exists Signed-off-by: Nikolas Komonen <nkomonen@amazon.com> * codewhisperer: method that indicates if valid connection exists For the add connection telemetry we want to know if a CW auth' connection already exists, not necessarily the currently active one. This adds 2 methods that return true if the valid CW connection exists and is known by the toolkit. Signed-off-by: Nikolas Komonen <nkomonen@amazon.com> * refactor: create isSsoConnection() alternatives A sso connection contains different types of sso connections, such as Builder ID or AWS Identity Center (Base). This commit creates functions to enable a user to check with specificity of if a connection is of a specific type of sso connection. Additionally, it updates existing uses to the appropriate implementation. Signed-off-by: Nikolas Komonen <nkomonen@amazon.com> * codecatalyst: function if toolkit is aware of valid builder id Signed-off-by: Nikolas Komonen <nkomonen@amazon.com> * refactor: fix circular dep issue Moves isValidCodeCatalystConnection() to the auth/connections.ts module and it fixes the circular dep Signed-off-by: Nikolas Komonen <nkomonen@amazon.com> * refactor: rename 'base' sso to 'idc' sso This naming is easier to understand even though the term Identity Center (idc) is purely a brand name and subject to change in the future. Signed-off-by: nkomonen <nkomonen@amazon.com> --------- Signed-off-by: Nikolas Komonen <nkomonen@amazon.com> Signed-off-by: nkomonen <nkomonen@amazon.com> * feat(cloudwatchlogs): preserve position while loading items #3934 Problem: When trying to select a CloudWatch Log Stream, the UI scrolls to the top while fetching all streams, which is very difficult to use. Solution: Set `keepScrollPosition = true` by default. Closes #2379 * codewhisperer: switch to default model when receiving ResouceNotFoundException (#3941) * filewatcher: template watcher lazy-loads if enableCodeLenses is off (#3938) ## Problem The template file watcher always ran regardless of SAM hint setting ## Solution The template file watcher now loads on demand. This appears to work even when toggling from off to on (codelenses appear on the template file) * ~~This may hit the startup function multiple times? Will do some more testing~~ should be resolved with 0b1df9d * Confirmed SAM debug works, when coming from startup with codelenses disabled * Confirmed the registry builds when codelenses are enabled * deps: bump @types/vscode to 1.68 #3880 previous: 5439f95 * refactor(watchedfiles): rename getRegisteredItem => getItem Problem: getRegisteredItem implies there is some other kind of "item", which is confusing. Solution: Rename to getItem. * refactor(watchedfiles): rename registeredItems => items Problem: The name `registeredItems` implies there is some other kind of "item", which is confusing and noisy, and unnecessarily verbose. Solution: Rename to `items`. * codewhisperer: reduce token infilling suggestions (#3944) * refactor(redshift): logging and output messages #3943 Problem: There are similar logging and output message in node explorer. Refactoring code to centralize logging and message output makes it easy for maintenance. Solution: Create message functions and add them in utils. * feat(lambda): increase timeout to 5 minutes #3954 * fix(sam): "Invalid (or missing) template file" Problem: SAM debug may fail to find template.yaml even if it is explicitly given by `invokeTarget.templatePath` in the vscode launch config. #2614 Solution: - Always call addItem() in validate(), instead of depending on the workspace-wide "scan". - Log a message when process() fails to process the file contents. * fix(sam): "Add SAM Debug Configuration" does nothing Problem: "AWS: Add SAM Debug Configuration" only works the first time it is invoked. regression from 89b8a94 Solution: Move samDebugConfigCmd() out of the if-condition. * changelog #3957 * cloud9: skip "!isWeb" check #3956 Problem: Cloud9 sets `isWeb` even though it has "compute". Solution: Add an explicit condition for Cloud9 wherever we check `isWeb`. ref d2f03d5 * feat(codewhisperer): select customization by prefix (#3947) Problem: The `aws.codeWhisperer.connect` command was introduced to allow enterprises to establish a default customization for onboarding developers after establishing an IdC connection. However, it only selected with a given ARN. Given the current behavior for adding updated customization for CodeWhisperer, an enterprise would need to keep the ARN up-to-date as well when calling this command. Solution: Provide the ability to define the customization to select with a prefix for the customization name. This way the logic to select a customization as updated customizations are introduced can be simplified on the administrator side by maintaining a prefix. e.g. MyModel-v1, MyModel-v2, etc. Co-authored-by: Will Lovo <wlovo@amazon.com> * perf(startup): avoid templateRegistry in awsFiletypes.ts #3962 Problem: The AWS Documents handler (`awsFiletypes.ts`) is triggered frequently. It calls `globals.templateRegistry` which triggers an expensive "Scanning CloudFormation templates..." process. #3510 Solution: Extract the validation logic out of `CloudFormationTemplateRegistry.process()` so that `awsFiletypes.ts` can use it without requesting the full registry. * Release 1.96.0 * Update version to snapshot version: 1.97.0-SNAPSHOT * feat: sm ide #3837 Problem: New SM VSCode IDE with IAM role, we want to use the IAM role for Code Whisperer auth. Currently Code Whisperer auth used Builder ID/SSO/Cloud9(IAM). Current IAM auth is for Cloud9 IDE and does not allow for paginated results. Solution: Add new IDE type for SM VSCode that can be used for this feature and future modifications. Use the new IDE type for conditional on connection type to use, defaulting to IAM. Use IAM for Code Whisperer and make sure we use the correct, `generateRecommendations`, endpoint to allow for auth with IAM and paginated results. --------- Signed-off-by: nkomonen <nkomonen@amazon.com> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Nikolas Komonen <nkomonen@amazon.com> Co-authored-by: Nikolas Komonen <118216176+nkomonen-amazon@users.noreply.github.com> Co-authored-by: Will Lo <96078566+Will-ShaoHua@users.noreply.github.com> Co-authored-by: Lei Gao <97199248+leigaol@users.noreply.github.com> Co-authored-by: Justin M. Keyes <jmkeyes@amazon.com> Co-authored-by: Bryce Ito <bryceito@amazon.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: yi-jin-0 <141669322+yi-jin-0@users.noreply.github.com> Co-authored-by: Zoe Lin <60411978+zixlin7@users.noreply.github.com> Co-authored-by: Santiago Martin <143631912+sannicm@users.noreply.github.com> Co-authored-by: Will Lovo <wlovo@users.noreply.github.com> Co-authored-by: Will Lovo <wlovo@amazon.com> Co-authored-by: aws-toolkit-automation <> Co-authored-by: Jed A <103950110+JedML@users.noreply.github.com> Co-authored-by: Tai Lai <ctlai95@gmail.com>
justinmk3
added a commit
that referenced
this issue
Nov 26, 2023
* CodeWhisperer: Security Issue Hover Provider (#3884) * add hover provider for security issues * add unit tests * codewhisperer: add markdown content for hover (#3890) * add markdown content for hover * allow multiple add/deletions in the same diff * codewhisperer: security hover update range and format (#3908) ## Problem Hovers positions are not being updated if the document changes. ## Solution Add an event handler which either removes the hover or offsets the line number depending on what was changed (similar to how [diagnostics are updated](https://github.com/aws/aws-toolkit-vscode/blob/0974becd8d26bc12a640b34bfbefc2e20fe8cb17/src/codewhisperer/service/diagnosticsProvider.ts#L61-L93 )) ## License By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. * codewhisperer: add quick fix code actions for issues (#3932) * codewhisperer: create webview panel for security issue (#3896) * create webview panel for security issue * update webview html * refactor to use vue webview factory * clean up * update webview styles * hide apply fix button when no suggested fix available * update model * fix tests * codewhisperer: telemetry metrics for hover #3955 Problem We want to track metrics for the number of times a user hovered on a vulnerability line. Solution Added metric event codewhisperer_codeScanIssueHover that will emit every time an issue is hovered over. * Merge master into feature/codewhisperer/assisted-code-remediation (#3929) * auth: ellipsis + right click options for `Developer Tools` (#3925) This commit allows user to right click to root CW or CC Node or click the new ellipsis icon, and they will get the options to Add Connection or Signout (if appropriate)   ## License By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. --------- Signed-off-by: nkomonen <nkomonen@amazon.com> * fix(codewihsperer): mapping language code before calling sendTelemetry API (#3928) * fix(codewhisperer): no auto trigger in Cloud9 #3927 * feat(settings): check that user settings.json is valid #3924 Problem: vscode notifies the user if `settings.json` is complete nonsense, but is silent if there are only "recoverable" JSON syntax errors. So _reading_ settings will appear to work, but _writing_ settings will throw an exception. #3910 Solution: On startup, check that we can actually _write_ settings, and show an error message if not. * watchedfiles: add watch patterns once to avoid rebuilding n times (#3931) Rrebuilding after each additional watch pattern meant we were processing globs n times, with each run containing `{1, 2, 3, ... , n-1, n}` patterns. Swapping this to an array and enforcing adding the patterns once ensures we only process 1 time, for n globs on initial load. We aren't adding globs dynamically at any point; if we ever feel the need to, we can implement some fancy caching/expiry business. FWIW it seems like getting the files in the glob is fairly fast, so if processing files is the bottleneck, this might be the cure? I'm sure we can improve more going forward regardless. <!--- REMINDER: - Read CONTRIBUTING.md first. - Add test coverage for your changes. - Update the changelog using `npm run newChange`. - Link to related issues/commits. - Testing: how did you test your changes? - Screenshots --> ## License By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. --------- Co-authored-by: Justin M. Keyes <jmkeyes@amazon.com> * test: skip e2e tests #3935 Reverts 7f98260 #3827 e2e tests are still always failing because: - webdriver html target is outdated - we are constantly throttled * build(deps): Bump the github-actions group with 1 update (#3937) Bumps the github-actions group with 1 update: [actions/setup-node](https://github.com/actions/setup-node). - [Release notes](https://github.com/actions/setup-node/releases) - [Commits](actions/setup-node@v3...v4) --- updated-dependencies: - dependency-name: actions/setup-node dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix(redshift): stale cache in connection wizard #3926 Problem: 1. When the connection parameters is removed, the re-launched connection wizard still has the earlier connection parameters. 2. When a sql query is successful but doesn't return any record, the output message is not clear. Solution: 1. Remove the connection wizard from the redshiftWarehouseNode constructor. When a connection wizard is launched, a new instance is created. 2. Modify the output message when a sql query is successful but doesn't return any record. * fix: Toolkit always opens Output panel on startup #3939 Problem: Output panel always opens at VS Code launch. #3930 Solution: Instead of showing Output panel at Redshift activation, change to show Output panel only when a message is added. * auth: Helper methods to check if connection exists (#3747) * auth: funcs to indicate if the connection exists Signed-off-by: Nikolas Komonen <nkomonen@amazon.com> * codewhisperer: method that indicates if valid connection exists For the add connection telemetry we want to know if a CW auth' connection already exists, not necessarily the currently active one. This adds 2 methods that return true if the valid CW connection exists and is known by the toolkit. Signed-off-by: Nikolas Komonen <nkomonen@amazon.com> * refactor: create isSsoConnection() alternatives A sso connection contains different types of sso connections, such as Builder ID or AWS Identity Center (Base). This commit creates functions to enable a user to check with specificity of if a connection is of a specific type of sso connection. Additionally, it updates existing uses to the appropriate implementation. Signed-off-by: Nikolas Komonen <nkomonen@amazon.com> * codecatalyst: function if toolkit is aware of valid builder id Signed-off-by: Nikolas Komonen <nkomonen@amazon.com> * refactor: fix circular dep issue Moves isValidCodeCatalystConnection() to the auth/connections.ts module and it fixes the circular dep Signed-off-by: Nikolas Komonen <nkomonen@amazon.com> * refactor: rename 'base' sso to 'idc' sso This naming is easier to understand even though the term Identity Center (idc) is purely a brand name and subject to change in the future. Signed-off-by: nkomonen <nkomonen@amazon.com> --------- Signed-off-by: Nikolas Komonen <nkomonen@amazon.com> Signed-off-by: nkomonen <nkomonen@amazon.com> * feat(cloudwatchlogs): preserve position while loading items #3934 Problem: When trying to select a CloudWatch Log Stream, the UI scrolls to the top while fetching all streams, which is very difficult to use. Solution: Set `keepScrollPosition = true` by default. Closes #2379 * codewhisperer: switch to default model when receiving ResouceNotFoundException (#3941) * filewatcher: template watcher lazy-loads if enableCodeLenses is off (#3938) ## Problem The template file watcher always ran regardless of SAM hint setting ## Solution The template file watcher now loads on demand. This appears to work even when toggling from off to on (codelenses appear on the template file) * ~~This may hit the startup function multiple times? Will do some more testing~~ should be resolved with 0b1df9d * Confirmed SAM debug works, when coming from startup with codelenses disabled * Confirmed the registry builds when codelenses are enabled * deps: bump @types/vscode to 1.68 #3880 previous: 5439f95 * refactor(watchedfiles): rename getRegisteredItem => getItem Problem: getRegisteredItem implies there is some other kind of "item", which is confusing. Solution: Rename to getItem. * refactor(watchedfiles): rename registeredItems => items Problem: The name `registeredItems` implies there is some other kind of "item", which is confusing and noisy, and unnecessarily verbose. Solution: Rename to `items`. * codewhisperer: reduce token infilling suggestions (#3944) * refactor(redshift): logging and output messages #3943 Problem: There are similar logging and output message in node explorer. Refactoring code to centralize logging and message output makes it easy for maintenance. Solution: Create message functions and add them in utils. * feat(lambda): increase timeout to 5 minutes #3954 * fix(sam): "Invalid (or missing) template file" Problem: SAM debug may fail to find template.yaml even if it is explicitly given by `invokeTarget.templatePath` in the vscode launch config. #2614 Solution: - Always call addItem() in validate(), instead of depending on the workspace-wide "scan". - Log a message when process() fails to process the file contents. * fix(sam): "Add SAM Debug Configuration" does nothing Problem: "AWS: Add SAM Debug Configuration" only works the first time it is invoked. regression from 89b8a94 Solution: Move samDebugConfigCmd() out of the if-condition. * changelog #3957 * cloud9: skip "!isWeb" check #3956 Problem: Cloud9 sets `isWeb` even though it has "compute". Solution: Add an explicit condition for Cloud9 wherever we check `isWeb`. ref d2f03d5 * feat(codewhisperer): select customization by prefix (#3947) Problem: The `aws.codeWhisperer.connect` command was introduced to allow enterprises to establish a default customization for onboarding developers after establishing an IdC connection. However, it only selected with a given ARN. Given the current behavior for adding updated customization for CodeWhisperer, an enterprise would need to keep the ARN up-to-date as well when calling this command. Solution: Provide the ability to define the customization to select with a prefix for the customization name. This way the logic to select a customization as updated customizations are introduced can be simplified on the administrator side by maintaining a prefix. e.g. MyModel-v1, MyModel-v2, etc. Co-authored-by: Will Lovo <wlovo@amazon.com> * perf(startup): avoid templateRegistry in awsFiletypes.ts #3962 Problem: The AWS Documents handler (`awsFiletypes.ts`) is triggered frequently. It calls `globals.templateRegistry` which triggers an expensive "Scanning CloudFormation templates..." process. #3510 Solution: Extract the validation logic out of `CloudFormationTemplateRegistry.process()` so that `awsFiletypes.ts` can use it without requesting the full registry. * Release 1.96.0 * Update version to snapshot version: 1.97.0-SNAPSHOT * feat: sm ide #3837 Problem: New SM VSCode IDE with IAM role, we want to use the IAM role for Code Whisperer auth. Currently Code Whisperer auth used Builder ID/SSO/Cloud9(IAM). Current IAM auth is for Cloud9 IDE and does not allow for paginated results. Solution: Add new IDE type for SM VSCode that can be used for this feature and future modifications. Use the new IDE type for conditional on connection type to use, defaulting to IAM. Use IAM for Code Whisperer and make sure we use the correct, `generateRecommendations`, endpoint to allow for auth with IAM and paginated results. --------- Signed-off-by: nkomonen <nkomonen@amazon.com> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Nikolas Komonen <nkomonen@amazon.com> Co-authored-by: Nikolas Komonen <118216176+nkomonen-amazon@users.noreply.github.com> Co-authored-by: Will Lo <96078566+Will-ShaoHua@users.noreply.github.com> Co-authored-by: Lei Gao <97199248+leigaol@users.noreply.github.com> Co-authored-by: Justin M. Keyes <jmkeyes@amazon.com> Co-authored-by: Bryce Ito <bryceito@amazon.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: yi-jin-0 <141669322+yi-jin-0@users.noreply.github.com> Co-authored-by: Zoe Lin <60411978+zixlin7@users.noreply.github.com> Co-authored-by: Santiago Martin <143631912+sannicm@users.noreply.github.com> Co-authored-by: Will Lovo <wlovo@users.noreply.github.com> Co-authored-by: Will Lovo <wlovo@amazon.com> Co-authored-by: aws-toolkit-automation <> Co-authored-by: Jed A <103950110+JedML@users.noreply.github.com> Co-authored-by: Tai Lai <ctlai95@gmail.com> * fix(codewhisperer): load hover images from dist folder (#3964) * fix(codewhisperer): load hover images from dist folder * fix tests * codewhisperer: emit issues with fixes count #3972 * feat(codewhisperer): apply security fix command (#3918) * Added apply security fix command * Added diff package to apply patch * added functionality to apply patch and update the file. * Wrote tests * feat(codewhisperer): code action to open webview (#3973) * feat(CodeWhisperer): add apply security fix command for quick fix and apply fix button in WebView (#3981) * add apply security fix command for quick fix and apply fix button in webview * updated text and action type for view details * codewhisperer: downgrade telemetry package (#4006) * codewhisperer: telemetry metrics apply fix (#4007) * codewhisperer: add metrics for apply fix * fix tests * move source values to component * use different component values for tests * revert move source values to component * feat(codewhisperer): add file path in webview (#4011) * added file path field in details to navigate back to the vulnerable line * fixed css for value of fixed available * codewhisperer: telemetry to use component metadata type (#4015) * codewhisperer: move images to resource folder (#4021) Severity svg images are not loading properly as icons, so creating a new folder `resources/images` instead * codewhisperer: open file in view column one (#4028) * codewhisperer: open file in view column one * return thenable * remove auto formatting * remove auto formatting * remove auto formatting * remove auto formatting --------- Signed-off-by: nkomonen <nkomonen@amazon.com> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Nikolas Komonen <nkomonen@amazon.com> Co-authored-by: aws-toolkit-automation <43144436+aws-toolkit-automation@users.noreply.github.com> Co-authored-by: Nikolas Komonen <118216176+nkomonen-amazon@users.noreply.github.com> Co-authored-by: Will Lo <96078566+Will-ShaoHua@users.noreply.github.com> Co-authored-by: Lei Gao <97199248+leigaol@users.noreply.github.com> Co-authored-by: Justin M. Keyes <jmkeyes@amazon.com> Co-authored-by: Bryce Ito <bryceito@amazon.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: yi-jin-0 <141669322+yi-jin-0@users.noreply.github.com> Co-authored-by: Zoe Lin <60411978+zixlin7@users.noreply.github.com> Co-authored-by: Santiago Martin <143631912+sannicm@users.noreply.github.com> Co-authored-by: Will Lovo <wlovo@users.noreply.github.com> Co-authored-by: Will Lovo <wlovo@amazon.com> Co-authored-by: Jed A <103950110+JedML@users.noreply.github.com> Co-authored-by: Manthan <manthanpatel.mj@gmail.com>
justinmk3
added a commit
that referenced
this issue
Nov 28, 2023
* CodeWhisperer: TS, C#, Cloud Formation[JSON & YAML/YML] & Terraform [TF & HCL] language support for Security Scans (#1182) * CW: Add TS language for CG Security scans * Add Csharp langauge for SecurityScans along with Test Cases * CW: Revised Code for C# language * CW: Adding CloudFormation support for security scans * CW:YAML and JSON implementation in VSC * CW:Test cases for Cloud Formation * CW:Test cases for CF * CW: Support for Terraform TF&HCL * CW: Add test cases for Terraform TF & HCL * CW: Revised Changes * Merge branch 'feature/iaacsecurityscans' staging into feature/iaacsecurityscans * CW: Revised changes for CSharp * CW: Changes in TS Implementation * CW: Change in HCL Implementation * CW: Revised changes Terraform * CW: Add TS language for CG Security scans * Add Csharp langauge for SecurityScans along with Test Cases * CW: Revised Code for C# language * CW: Adding CloudFormation support for security scans * CW:YAML and JSON implementation in VSC * CW:Test cases for Cloud Formation * CW:Test cases for CF * CW: Support for Terraform TF&HCL * CW: Add test cases for Terraform TF & HCL * CW: Revised Changes * Merge branch 'feature/iaacsecurityscans' staging into feature/iaacsecurityscans * CW: Revised changes for CSharp * CW: Changes in TS Implementation * CW: Change in HCL Implementation * CW: Revised changes Terraform * External fix(ecs): exec command requires SimulatePrincipalPolicy permissions #3993 Problem: We attempt `SimulatePrincipalPolicy` to verify the container task role has proper permissions before running exec command. Users may not have access to simulate policies, but may otherwise be able to run exec command without issue. #3969 Solution: Only attempt `SimulatePrincipalPolicy` if exec command fails. Otherwise, there is no need to simulate policies. * CW: Revised changes * CW:Revised Code * CW:Revised Changes according to PR Comments * CW:Revised Changes * CW: Various Changes * CW: various changes according to PR * CW: Adding a Test Case for CSharp Security Scans --------- Co-authored-by: Laxman Reddy <141967714+LaxmanReddyAileni@users.noreply.github.com> * Merge from public (#1275) * CodeWhisperer: Security Issue Hover Provider (#3884) * add hover provider for security issues * add unit tests * codewhisperer: add markdown content for hover (#3890) * add markdown content for hover * allow multiple add/deletions in the same diff * codewhisperer: security hover update range and format (#3908) ## Problem Hovers positions are not being updated if the document changes. ## Solution Add an event handler which either removes the hover or offsets the line number depending on what was changed (similar to how [diagnostics are updated](https://github.com/aws/aws-toolkit-vscode/blob/0974becd8d26bc12a640b34bfbefc2e20fe8cb17/src/codewhisperer/service/diagnosticsProvider.ts#L61-L93 )) ## License By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. * codewhisperer: add quick fix code actions for issues (#3932) * codewhisperer: create webview panel for security issue (#3896) * create webview panel for security issue * update webview html * refactor to use vue webview factory * clean up * update webview styles * hide apply fix button when no suggested fix available * update model * fix tests * codewhisperer: telemetry metrics for hover #3955 Problem We want to track metrics for the number of times a user hovered on a vulnerability line. Solution Added metric event codewhisperer_codeScanIssueHover that will emit every time an issue is hovered over. * Merge master into feature/codewhisperer/assisted-code-remediation (#3929) * auth: ellipsis + right click options for `Developer Tools` (#3925) This commit allows user to right click to root CW or CC Node or click the new ellipsis icon, and they will get the options to Add Connection or Signout (if appropriate)   ## License By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. --------- Signed-off-by: nkomonen <nkomonen@amazon.com> * fix(codewihsperer): mapping language code before calling sendTelemetry API (#3928) * fix(codewhisperer): no auto trigger in Cloud9 #3927 * feat(settings): check that user settings.json is valid #3924 Problem: vscode notifies the user if `settings.json` is complete nonsense, but is silent if there are only "recoverable" JSON syntax errors. So _reading_ settings will appear to work, but _writing_ settings will throw an exception. #3910 Solution: On startup, check that we can actually _write_ settings, and show an error message if not. * watchedfiles: add watch patterns once to avoid rebuilding n times (#3931) Rrebuilding after each additional watch pattern meant we were processing globs n times, with each run containing `{1, 2, 3, ... , n-1, n}` patterns. Swapping this to an array and enforcing adding the patterns once ensures we only process 1 time, for n globs on initial load. We aren't adding globs dynamically at any point; if we ever feel the need to, we can implement some fancy caching/expiry business. FWIW it seems like getting the files in the glob is fairly fast, so if processing files is the bottleneck, this might be the cure? I'm sure we can improve more going forward regardless. <!--- REMINDER: - Read CONTRIBUTING.md first. - Add test coverage for your changes. - Update the changelog using `npm run newChange`. - Link to related issues/commits. - Testing: how did you test your changes? - Screenshots --> ## License By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. --------- Co-authored-by: Justin M. Keyes <jmkeyes@amazon.com> * test: skip e2e tests #3935 Reverts 7f98260 #3827 e2e tests are still always failing because: - webdriver html target is outdated - we are constantly throttled * build(deps): Bump the github-actions group with 1 update (#3937) Bumps the github-actions group with 1 update: [actions/setup-node](https://github.com/actions/setup-node). - [Release notes](https://github.com/actions/setup-node/releases) - [Commits](actions/setup-node@v3...v4) --- updated-dependencies: - dependency-name: actions/setup-node dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix(redshift): stale cache in connection wizard #3926 Problem: 1. When the connection parameters is removed, the re-launched connection wizard still has the earlier connection parameters. 2. When a sql query is successful but doesn't return any record, the output message is not clear. Solution: 1. Remove the connection wizard from the redshiftWarehouseNode constructor. When a connection wizard is launched, a new instance is created. 2. Modify the output message when a sql query is successful but doesn't return any record. * fix: Toolkit always opens Output panel on startup #3939 Problem: Output panel always opens at VS Code launch. #3930 Solution: Instead of showing Output panel at Redshift activation, change to show Output panel only when a message is added. * auth: Helper methods to check if connection exists (#3747) * auth: funcs to indicate if the connection exists Signed-off-by: Nikolas Komonen <nkomonen@amazon.com> * codewhisperer: method that indicates if valid connection exists For the add connection telemetry we want to know if a CW auth' connection already exists, not necessarily the currently active one. This adds 2 methods that return true if the valid CW connection exists and is known by the toolkit. Signed-off-by: Nikolas Komonen <nkomonen@amazon.com> * refactor: create isSsoConnection() alternatives A sso connection contains different types of sso connections, such as Builder ID or AWS Identity Center (Base). This commit creates functions to enable a user to check with specificity of if a connection is of a specific type of sso connection. Additionally, it updates existing uses to the appropriate implementation. Signed-off-by: Nikolas Komonen <nkomonen@amazon.com> * codecatalyst: function if toolkit is aware of valid builder id Signed-off-by: Nikolas Komonen <nkomonen@amazon.com> * refactor: fix circular dep issue Moves isValidCodeCatalystConnection() to the auth/connections.ts module and it fixes the circular dep Signed-off-by: Nikolas Komonen <nkomonen@amazon.com> * refactor: rename 'base' sso to 'idc' sso This naming is easier to understand even though the term Identity Center (idc) is purely a brand name and subject to change in the future. Signed-off-by: nkomonen <nkomonen@amazon.com> --------- Signed-off-by: Nikolas Komonen <nkomonen@amazon.com> Signed-off-by: nkomonen <nkomonen@amazon.com> * feat(cloudwatchlogs): preserve position while loading items #3934 Problem: When trying to select a CloudWatch Log Stream, the UI scrolls to the top while fetching all streams, which is very difficult to use. Solution: Set `keepScrollPosition = true` by default. Closes #2379 * codewhisperer: switch to default model when receiving ResouceNotFoundException (#3941) * filewatcher: template watcher lazy-loads if enableCodeLenses is off (#3938) ## Problem The template file watcher always ran regardless of SAM hint setting ## Solution The template file watcher now loads on demand. This appears to work even when toggling from off to on (codelenses appear on the template file) * ~~This may hit the startup function multiple times? Will do some more testing~~ should be resolved with 0b1df9d * Confirmed SAM debug works, when coming from startup with codelenses disabled * Confirmed the registry builds when codelenses are enabled * deps: bump @types/vscode to 1.68 #3880 previous: 5439f95 * refactor(watchedfiles): rename getRegisteredItem => getItem Problem: getRegisteredItem implies there is some other kind of "item", which is confusing. Solution: Rename to getItem. * refactor(watchedfiles): rename registeredItems => items Problem: The name `registeredItems` implies there is some other kind of "item", which is confusing and noisy, and unnecessarily verbose. Solution: Rename to `items`. * codewhisperer: reduce token infilling suggestions (#3944) * refactor(redshift): logging and output messages #3943 Problem: There are similar logging and output message in node explorer. Refactoring code to centralize logging and message output makes it easy for maintenance. Solution: Create message functions and add them in utils. * feat(lambda): increase timeout to 5 minutes #3954 * fix(sam): "Invalid (or missing) template file" Problem: SAM debug may fail to find template.yaml even if it is explicitly given by `invokeTarget.templatePath` in the vscode launch config. #2614 Solution: - Always call addItem() in validate(), instead of depending on the workspace-wide "scan". - Log a message when process() fails to process the file contents. * fix(sam): "Add SAM Debug Configuration" does nothing Problem: "AWS: Add SAM Debug Configuration" only works the first time it is invoked. regression from 89b8a94 Solution: Move samDebugConfigCmd() out of the if-condition. * changelog #3957 * cloud9: skip "!isWeb" check #3956 Problem: Cloud9 sets `isWeb` even though it has "compute". Solution: Add an explicit condition for Cloud9 wherever we check `isWeb`. ref d2f03d5 * feat(codewhisperer): select customization by prefix (#3947) Problem: The `aws.codeWhisperer.connect` command was introduced to allow enterprises to establish a default customization for onboarding developers after establishing an IdC connection. However, it only selected with a given ARN. Given the current behavior for adding updated customization for CodeWhisperer, an enterprise would need to keep the ARN up-to-date as well when calling this command. Solution: Provide the ability to define the customization to select with a prefix for the customization name. This way the logic to select a customization as updated customizations are introduced can be simplified on the administrator side by maintaining a prefix. e.g. MyModel-v1, MyModel-v2, etc. Co-authored-by: Will Lovo <wlovo@amazon.com> * perf(startup): avoid templateRegistry in awsFiletypes.ts #3962 Problem: The AWS Documents handler (`awsFiletypes.ts`) is triggered frequently. It calls `globals.templateRegistry` which triggers an expensive "Scanning CloudFormation templates..." process. #3510 Solution: Extract the validation logic out of `CloudFormationTemplateRegistry.process()` so that `awsFiletypes.ts` can use it without requesting the full registry. * Release 1.96.0 * Update version to snapshot version: 1.97.0-SNAPSHOT * feat: sm ide #3837 Problem: New SM VSCode IDE with IAM role, we want to use the IAM role for Code Whisperer auth. Currently Code Whisperer auth used Builder ID/SSO/Cloud9(IAM). Current IAM auth is for Cloud9 IDE and does not allow for paginated results. Solution: Add new IDE type for SM VSCode that can be used for this feature and future modifications. Use the new IDE type for conditional on connection type to use, defaulting to IAM. Use IAM for Code Whisperer and make sure we use the correct, `generateRecommendations`, endpoint to allow for auth with IAM and paginated results. --------- Signed-off-by: nkomonen <nkomonen@amazon.com> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Nikolas Komonen <nkomonen@amazon.com> Co-authored-by: Nikolas Komonen <118216176+nkomonen-amazon@users.noreply.github.com> Co-authored-by: Will Lo <96078566+Will-ShaoHua@users.noreply.github.com> Co-authored-by: Lei Gao <97199248+leigaol@users.noreply.github.com> Co-authored-by: Justin M. Keyes <jmkeyes@amazon.com> Co-authored-by: Bryce Ito <bryceito@amazon.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: yi-jin-0 <141669322+yi-jin-0@users.noreply.github.com> Co-authored-by: Zoe Lin <60411978+zixlin7@users.noreply.github.com> Co-authored-by: Santiago Martin <143631912+sannicm@users.noreply.github.com> Co-authored-by: Will Lovo <wlovo@users.noreply.github.com> Co-authored-by: Will Lovo <wlovo@amazon.com> Co-authored-by: aws-toolkit-automation <> Co-authored-by: Jed A <103950110+JedML@users.noreply.github.com> Co-authored-by: Tai Lai <ctlai95@gmail.com> * fix(codewhisperer): load hover images from dist folder (#3964) * fix(codewhisperer): load hover images from dist folder * fix tests * codewhisperer: emit issues with fixes count #3972 * feat(codewhisperer): apply security fix command (#3918) * Added apply security fix command * Added diff package to apply patch * added functionality to apply patch and update the file. * Wrote tests * feat(codewhisperer): code action to open webview (#3973) * feat(CodeWhisperer): add apply security fix command for quick fix and apply fix button in WebView (#3981) * add apply security fix command for quick fix and apply fix button in webview * updated text and action type for view details * codewhisperer: downgrade telemetry package (#4006) * codewhisperer: telemetry metrics apply fix (#4007) * codewhisperer: add metrics for apply fix * fix tests * move source values to component * use different component values for tests * revert move source values to component * feat(codewhisperer): add file path in webview (#4011) * added file path field in details to navigate back to the vulnerable line * fixed css for value of fixed available * codewhisperer: telemetry to use component metadata type (#4015) * codewhisperer: move images to resource folder (#4021) Severity svg images are not loading properly as icons, so creating a new folder `resources/images` instead * codewhisperer: open file in view column one (#4028) * codewhisperer: open file in view column one * return thenable * remove auto formatting * remove auto formatting * remove auto formatting * remove auto formatting --------- Signed-off-by: nkomonen <nkomonen@amazon.com> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Nikolas Komonen <nkomonen@amazon.com> Co-authored-by: aws-toolkit-automation <43144436+aws-toolkit-automation@users.noreply.github.com> Co-authored-by: Nikolas Komonen <118216176+nkomonen-amazon@users.noreply.github.com> Co-authored-by: Will Lo <96078566+Will-ShaoHua@users.noreply.github.com> Co-authored-by: Lei Gao <97199248+leigaol@users.noreply.github.com> Co-authored-by: Justin M. Keyes <jmkeyes@amazon.com> Co-authored-by: Bryce Ito <bryceito@amazon.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: yi-jin-0 <141669322+yi-jin-0@users.noreply.github.com> Co-authored-by: Zoe Lin <60411978+zixlin7@users.noreply.github.com> Co-authored-by: Santiago Martin <143631912+sannicm@users.noreply.github.com> Co-authored-by: Will Lovo <wlovo@users.noreply.github.com> Co-authored-by: Will Lovo <wlovo@amazon.com> Co-authored-by: Jed A <103950110+JedML@users.noreply.github.com> Co-authored-by: Manthan <manthanpatel.mj@gmail.com> * CW: Adding languages to runtimeLanguageSet for Telemetry in Security Scans (#1272) * Changes in Telemetry * type check chnage --------- Co-authored-by: Laxman Reddy <141967714+LaxmanReddyAileni@users.noreply.github.com> * Merging staging to feature/codewhisperer/asssited-coderemediation branch (#1305) perf(codewhisperer): connection reuse for generationCompletions #4040 Problem: [microsoft/vscode#173861](microsoft/vscode#173861) Solution: Inject http agent only for the GenerateCompletions API, do this per IDE session. Tested and verified: 1. This inject won't happen for any other API calls. It only applies to GenerateCompletions API when user is NOT using `http.proxy` vscode feature. 2. No regression found with Toolkit features or CodeWhisperer features. 3. Regardless of user VSC proxy setting `http.proxySupport`, it still enables HTTP connection reuse. 4. Tested in AWS Cloud9. Co-authored-by: Lei Gao <97199248+leigaol@users.noreply.github.com> * fix(codewhisperer): hide acr description if empty Problem: Backend is returning empty string or `Suggested remediation:` for some remediation description, which causes incomplete sections in the UI. Solution: Handle this case by hiding "Why are we recommending this?" section if description is missing/incomplete. * CW: Adding selected JSON File into zip for the Security Scan Problem Zip file for security scan does not contain selected file. Solution Fixed this in this PR. Co-authored-by: Laxman Reddy <141967714+LaxmanReddyAileni@users.noreply.github.com> --------- Signed-off-by: nkomonen <nkomonen@amazon.com> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Nikolas Komonen <nkomonen@amazon.com> Co-authored-by: aws-toolkit-automation <43144436+aws-toolkit-automation@users.noreply.github.com> Co-authored-by: Laxman Reddy <141967714+laileni-aws@users.noreply.github.com> Co-authored-by: Laxman Reddy <141967714+LaxmanReddyAileni@users.noreply.github.com> Co-authored-by: Nikolas Komonen <118216176+nkomonen-amazon@users.noreply.github.com> Co-authored-by: Will Lo <96078566+Will-ShaoHua@users.noreply.github.com> Co-authored-by: Lei Gao <97199248+leigaol@users.noreply.github.com> Co-authored-by: Justin M. Keyes <jmkeyes@amazon.com> Co-authored-by: Bryce Ito <bryceito@amazon.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: yi-jin-0 <141669322+yi-jin-0@users.noreply.github.com> Co-authored-by: Zoe Lin <60411978+zixlin7@users.noreply.github.com> Co-authored-by: Santiago Martin <143631912+sannicm@users.noreply.github.com> Co-authored-by: Will Lovo <wlovo@users.noreply.github.com> Co-authored-by: Will Lovo <wlovo@amazon.com> Co-authored-by: Jed A <103950110+JedML@users.noreply.github.com> Co-authored-by: Manthan <manthanpatel.mj@gmail.com>
justinmk3
added a commit
that referenced
this issue
Nov 28, 2023
* CodeWhisperer: TS, C#, Cloud Formation[JSON & YAML/YML] & Terraform [TF & HCL] language support for Security Scans (#1182) * CW: Add TS language for CG Security scans * Add Csharp langauge for SecurityScans along with Test Cases * CW: Revised Code for C# language * CW: Adding CloudFormation support for security scans * CW:YAML and JSON implementation in VSC * CW:Test cases for Cloud Formation * CW:Test cases for CF * CW: Support for Terraform TF&HCL * CW: Add test cases for Terraform TF & HCL * CW: Revised Changes * Merge branch 'feature/iaacsecurityscans' staging into feature/iaacsecurityscans * CW: Revised changes for CSharp * CW: Changes in TS Implementation * CW: Change in HCL Implementation * CW: Revised changes Terraform * CW: Add TS language for CG Security scans * Add Csharp langauge for SecurityScans along with Test Cases * CW: Revised Code for C# language * CW: Adding CloudFormation support for security scans * CW:YAML and JSON implementation in VSC * CW:Test cases for Cloud Formation * CW:Test cases for CF * CW: Support for Terraform TF&HCL * CW: Add test cases for Terraform TF & HCL * CW: Revised Changes * Merge branch 'feature/iaacsecurityscans' staging into feature/iaacsecurityscans * CW: Revised changes for CSharp * CW: Changes in TS Implementation * CW: Change in HCL Implementation * CW: Revised changes Terraform * External fix(ecs): exec command requires SimulatePrincipalPolicy permissions #3993 Problem: We attempt `SimulatePrincipalPolicy` to verify the container task role has proper permissions before running exec command. Users may not have access to simulate policies, but may otherwise be able to run exec command without issue. #3969 Solution: Only attempt `SimulatePrincipalPolicy` if exec command fails. Otherwise, there is no need to simulate policies. * CW: Revised changes * CW:Revised Code * CW:Revised Changes according to PR Comments * CW:Revised Changes * CW: Various Changes * CW: various changes according to PR * CW: Adding a Test Case for CSharp Security Scans --------- Co-authored-by: Laxman Reddy <141967714+LaxmanReddyAileni@users.noreply.github.com> * fix(codewhisperer): Add connection reuse for generationCompletions (#1214) * Add connection reuse for generationCompletions * Merge from public (#1275) * CodeWhisperer: Security Issue Hover Provider (#3884) * add hover provider for security issues * add unit tests * codewhisperer: add markdown content for hover (#3890) * add markdown content for hover * allow multiple add/deletions in the same diff * codewhisperer: security hover update range and format (#3908) ## Problem Hovers positions are not being updated if the document changes. ## Solution Add an event handler which either removes the hover or offsets the line number depending on what was changed (similar to how [diagnostics are updated](https://github.com/aws/aws-toolkit-vscode/blob/0974becd8d26bc12a640b34bfbefc2e20fe8cb17/src/codewhisperer/service/diagnosticsProvider.ts#L61-L93 )) ## License By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. * codewhisperer: add quick fix code actions for issues (#3932) * codewhisperer: create webview panel for security issue (#3896) * create webview panel for security issue * update webview html * refactor to use vue webview factory * clean up * update webview styles * hide apply fix button when no suggested fix available * update model * fix tests * codewhisperer: telemetry metrics for hover #3955 Problem We want to track metrics for the number of times a user hovered on a vulnerability line. Solution Added metric event codewhisperer_codeScanIssueHover that will emit every time an issue is hovered over. * Merge master into feature/codewhisperer/assisted-code-remediation (#3929) * auth: ellipsis + right click options for `Developer Tools` (#3925) This commit allows user to right click to root CW or CC Node or click the new ellipsis icon, and they will get the options to Add Connection or Signout (if appropriate)   ## License By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. --------- Signed-off-by: nkomonen <nkomonen@amazon.com> * fix(codewihsperer): mapping language code before calling sendTelemetry API (#3928) * fix(codewhisperer): no auto trigger in Cloud9 #3927 * feat(settings): check that user settings.json is valid #3924 Problem: vscode notifies the user if `settings.json` is complete nonsense, but is silent if there are only "recoverable" JSON syntax errors. So _reading_ settings will appear to work, but _writing_ settings will throw an exception. #3910 Solution: On startup, check that we can actually _write_ settings, and show an error message if not. * watchedfiles: add watch patterns once to avoid rebuilding n times (#3931) Rrebuilding after each additional watch pattern meant we were processing globs n times, with each run containing `{1, 2, 3, ... , n-1, n}` patterns. Swapping this to an array and enforcing adding the patterns once ensures we only process 1 time, for n globs on initial load. We aren't adding globs dynamically at any point; if we ever feel the need to, we can implement some fancy caching/expiry business. FWIW it seems like getting the files in the glob is fairly fast, so if processing files is the bottleneck, this might be the cure? I'm sure we can improve more going forward regardless. <!--- REMINDER: - Read CONTRIBUTING.md first. - Add test coverage for your changes. - Update the changelog using `npm run newChange`. - Link to related issues/commits. - Testing: how did you test your changes? - Screenshots --> ## License By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. --------- Co-authored-by: Justin M. Keyes <jmkeyes@amazon.com> * test: skip e2e tests #3935 Reverts 7f98260 #3827 e2e tests are still always failing because: - webdriver html target is outdated - we are constantly throttled * build(deps): Bump the github-actions group with 1 update (#3937) Bumps the github-actions group with 1 update: [actions/setup-node](https://github.com/actions/setup-node). - [Release notes](https://github.com/actions/setup-node/releases) - [Commits](actions/setup-node@v3...v4) --- updated-dependencies: - dependency-name: actions/setup-node dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix(redshift): stale cache in connection wizard #3926 Problem: 1. When the connection parameters is removed, the re-launched connection wizard still has the earlier connection parameters. 2. When a sql query is successful but doesn't return any record, the output message is not clear. Solution: 1. Remove the connection wizard from the redshiftWarehouseNode constructor. When a connection wizard is launched, a new instance is created. 2. Modify the output message when a sql query is successful but doesn't return any record. * fix: Toolkit always opens Output panel on startup #3939 Problem: Output panel always opens at VS Code launch. #3930 Solution: Instead of showing Output panel at Redshift activation, change to show Output panel only when a message is added. * auth: Helper methods to check if connection exists (#3747) * auth: funcs to indicate if the connection exists Signed-off-by: Nikolas Komonen <nkomonen@amazon.com> * codewhisperer: method that indicates if valid connection exists For the add connection telemetry we want to know if a CW auth' connection already exists, not necessarily the currently active one. This adds 2 methods that return true if the valid CW connection exists and is known by the toolkit. Signed-off-by: Nikolas Komonen <nkomonen@amazon.com> * refactor: create isSsoConnection() alternatives A sso connection contains different types of sso connections, such as Builder ID or AWS Identity Center (Base). This commit creates functions to enable a user to check with specificity of if a connection is of a specific type of sso connection. Additionally, it updates existing uses to the appropriate implementation. Signed-off-by: Nikolas Komonen <nkomonen@amazon.com> * codecatalyst: function if toolkit is aware of valid builder id Signed-off-by: Nikolas Komonen <nkomonen@amazon.com> * refactor: fix circular dep issue Moves isValidCodeCatalystConnection() to the auth/connections.ts module and it fixes the circular dep Signed-off-by: Nikolas Komonen <nkomonen@amazon.com> * refactor: rename 'base' sso to 'idc' sso This naming is easier to understand even though the term Identity Center (idc) is purely a brand name and subject to change in the future. Signed-off-by: nkomonen <nkomonen@amazon.com> --------- Signed-off-by: Nikolas Komonen <nkomonen@amazon.com> Signed-off-by: nkomonen <nkomonen@amazon.com> * feat(cloudwatchlogs): preserve position while loading items #3934 Problem: When trying to select a CloudWatch Log Stream, the UI scrolls to the top while fetching all streams, which is very difficult to use. Solution: Set `keepScrollPosition = true` by default. Closes #2379 * codewhisperer: switch to default model when receiving ResouceNotFoundException (#3941) * filewatcher: template watcher lazy-loads if enableCodeLenses is off (#3938) ## Problem The template file watcher always ran regardless of SAM hint setting ## Solution The template file watcher now loads on demand. This appears to work even when toggling from off to on (codelenses appear on the template file) * ~~This may hit the startup function multiple times? Will do some more testing~~ should be resolved with 0b1df9d * Confirmed SAM debug works, when coming from startup with codelenses disabled * Confirmed the registry builds when codelenses are enabled * deps: bump @types/vscode to 1.68 #3880 previous: 5439f95 * refactor(watchedfiles): rename getRegisteredItem => getItem Problem: getRegisteredItem implies there is some other kind of "item", which is confusing. Solution: Rename to getItem. * refactor(watchedfiles): rename registeredItems => items Problem: The name `registeredItems` implies there is some other kind of "item", which is confusing and noisy, and unnecessarily verbose. Solution: Rename to `items`. * codewhisperer: reduce token infilling suggestions (#3944) * refactor(redshift): logging and output messages #3943 Problem: There are similar logging and output message in node explorer. Refactoring code to centralize logging and message output makes it easy for maintenance. Solution: Create message functions and add them in utils. * feat(lambda): increase timeout to 5 minutes #3954 * fix(sam): "Invalid (or missing) template file" Problem: SAM debug may fail to find template.yaml even if it is explicitly given by `invokeTarget.templatePath` in the vscode launch config. #2614 Solution: - Always call addItem() in validate(), instead of depending on the workspace-wide "scan". - Log a message when process() fails to process the file contents. * fix(sam): "Add SAM Debug Configuration" does nothing Problem: "AWS: Add SAM Debug Configuration" only works the first time it is invoked. regression from 89b8a94 Solution: Move samDebugConfigCmd() out of the if-condition. * changelog #3957 * cloud9: skip "!isWeb" check #3956 Problem: Cloud9 sets `isWeb` even though it has "compute". Solution: Add an explicit condition for Cloud9 wherever we check `isWeb`. ref d2f03d5 * feat(codewhisperer): select customization by prefix (#3947) Problem: The `aws.codeWhisperer.connect` command was introduced to allow enterprises to establish a default customization for onboarding developers after establishing an IdC connection. However, it only selected with a given ARN. Given the current behavior for adding updated customization for CodeWhisperer, an enterprise would need to keep the ARN up-to-date as well when calling this command. Solution: Provide the ability to define the customization to select with a prefix for the customization name. This way the logic to select a customization as updated customizations are introduced can be simplified on the administrator side by maintaining a prefix. e.g. MyModel-v1, MyModel-v2, etc. Co-authored-by: Will Lovo <wlovo@amazon.com> * perf(startup): avoid templateRegistry in awsFiletypes.ts #3962 Problem: The AWS Documents handler (`awsFiletypes.ts`) is triggered frequently. It calls `globals.templateRegistry` which triggers an expensive "Scanning CloudFormation templates..." process. #3510 Solution: Extract the validation logic out of `CloudFormationTemplateRegistry.process()` so that `awsFiletypes.ts` can use it without requesting the full registry. * Release 1.96.0 * Update version to snapshot version: 1.97.0-SNAPSHOT * feat: sm ide #3837 Problem: New SM VSCode IDE with IAM role, we want to use the IAM role for Code Whisperer auth. Currently Code Whisperer auth used Builder ID/SSO/Cloud9(IAM). Current IAM auth is for Cloud9 IDE and does not allow for paginated results. Solution: Add new IDE type for SM VSCode that can be used for this feature and future modifications. Use the new IDE type for conditional on connection type to use, defaulting to IAM. Use IAM for Code Whisperer and make sure we use the correct, `generateRecommendations`, endpoint to allow for auth with IAM and paginated results. --------- Signed-off-by: nkomonen <nkomonen@amazon.com> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Nikolas Komonen <nkomonen@amazon.com> Co-authored-by: Nikolas Komonen <118216176+nkomonen-amazon@users.noreply.github.com> Co-authored-by: Will Lo <96078566+Will-ShaoHua@users.noreply.github.com> Co-authored-by: Lei Gao <97199248+leigaol@users.noreply.github.com> Co-authored-by: Justin M. Keyes <jmkeyes@amazon.com> Co-authored-by: Bryce Ito <bryceito@amazon.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: yi-jin-0 <141669322+yi-jin-0@users.noreply.github.com> Co-authored-by: Zoe Lin <60411978+zixlin7@users.noreply.github.com> Co-authored-by: Santiago Martin <143631912+sannicm@users.noreply.github.com> Co-authored-by: Will Lovo <wlovo@users.noreply.github.com> Co-authored-by: Will Lovo <wlovo@amazon.com> Co-authored-by: aws-toolkit-automation <> Co-authored-by: Jed A <103950110+JedML@users.noreply.github.com> Co-authored-by: Tai Lai <ctlai95@gmail.com> * fix(codewhisperer): load hover images from dist folder (#3964) * fix(codewhisperer): load hover images from dist folder * fix tests * codewhisperer: emit issues with fixes count #3972 * feat(codewhisperer): apply security fix command (#3918) * Added apply security fix command * Added diff package to apply patch * added functionality to apply patch and update the file. * Wrote tests * feat(codewhisperer): code action to open webview (#3973) * feat(CodeWhisperer): add apply security fix command for quick fix and apply fix button in WebView (#3981) * add apply security fix command for quick fix and apply fix button in webview * updated text and action type for view details * codewhisperer: downgrade telemetry package (#4006) * codewhisperer: telemetry metrics apply fix (#4007) * codewhisperer: add metrics for apply fix * fix tests * move source values to component * use different component values for tests * revert move source values to component * feat(codewhisperer): add file path in webview (#4011) * added file path field in details to navigate back to the vulnerable line * fixed css for value of fixed available * codewhisperer: telemetry to use component metadata type (#4015) * codewhisperer: move images to resource folder (#4021) Severity svg images are not loading properly as icons, so creating a new folder `resources/images` instead * codewhisperer: open file in view column one (#4028) * codewhisperer: open file in view column one * return thenable * remove auto formatting * remove auto formatting * remove auto formatting * remove auto formatting --------- Signed-off-by: nkomonen <nkomonen@amazon.com> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Nikolas Komonen <nkomonen@amazon.com> Co-authored-by: aws-toolkit-automation <43144436+aws-toolkit-automation@users.noreply.github.com> Co-authored-by: Nikolas Komonen <118216176+nkomonen-amazon@users.noreply.github.com> Co-authored-by: Will Lo <96078566+Will-ShaoHua@users.noreply.github.com> Co-authored-by: Lei Gao <97199248+leigaol@users.noreply.github.com> Co-authored-by: Justin M. Keyes <jmkeyes@amazon.com> Co-authored-by: Bryce Ito <bryceito@amazon.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: yi-jin-0 <141669322+yi-jin-0@users.noreply.github.com> Co-authored-by: Zoe Lin <60411978+zixlin7@users.noreply.github.com> Co-authored-by: Santiago Martin <143631912+sannicm@users.noreply.github.com> Co-authored-by: Will Lovo <wlovo@users.noreply.github.com> Co-authored-by: Will Lovo <wlovo@amazon.com> Co-authored-by: Jed A <103950110+JedML@users.noreply.github.com> Co-authored-by: Manthan <manthanpatel.mj@gmail.com> * CW: Adding languages to runtimeLanguageSet for Telemetry in Security Scans (#1272) * Changes in Telemetry * type check chnage --------- Co-authored-by: Laxman Reddy <141967714+LaxmanReddyAileni@users.noreply.github.com> * Merging staging to feature/codewhisperer/asssited-coderemediation branch (#1305) perf(codewhisperer): connection reuse for generationCompletions #4040 Problem: [microsoft/vscode#173861](microsoft/vscode#173861) Solution: Inject http agent only for the GenerateCompletions API, do this per IDE session. Tested and verified: 1. This inject won't happen for any other API calls. It only applies to GenerateCompletions API when user is NOT using `http.proxy` vscode feature. 2. No regression found with Toolkit features or CodeWhisperer features. 3. Regardless of user VSC proxy setting `http.proxySupport`, it still enables HTTP connection reuse. 4. Tested in AWS Cloud9. Co-authored-by: Lei Gao <97199248+leigaol@users.noreply.github.com> * fix(codewhisperer): hide acr description if empty Problem: Backend is returning empty string or `Suggested remediation:` for some remediation description, which causes incomplete sections in the UI. Solution: Handle this case by hiding "Why are we recommending this?" section if description is missing/incomplete. * CW: Adding selected JSON File into zip for the Security Scan Problem Zip file for security scan does not contain selected file. Solution Fixed this in this PR. Co-authored-by: Laxman Reddy <141967714+LaxmanReddyAileni@users.noreply.github.com> * CW: Terraform is supported using other ext extensions (#1380) * CW: Terraform is supported using other ext extensions * CW: ADD test cases --------- Co-authored-by: Laxman Reddy <141967714+LaxmanReddyAileni@users.noreply.github.com> * codewhisperer: update styles for hover and webview (#1384) * codewhisperer: update styles for hover and webview * update padding * hide no newline indicator * Iac language expansion for VSCode (#1352) * Changes to add new languages json, yaml and tf. * hds * sdch * changes for adding new languages json, yaml and tf * changing endpoint to gamma IAD * revisions for Iac Language Support for Json, Yaml and Tf along with added Unit Test * correction for endpoint and added comments for language Ids * corrections for unit tests to add plaintext * updating telemetry for languages * fix build failure * fix pkg * Add change log for security scan new language support (#1418) Change log Co-authored-by: Laxman Reddy <141967714+LaxmanReddyAileni@users.noreply.github.com> * codewhisperer: update code diff styles (#1407) * codewhisperer: update code diff styles * add change log * remove extra release notes --------- Signed-off-by: nkomonen <nkomonen@amazon.com> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Nikolas Komonen <nkomonen@amazon.com> Co-authored-by: aws-toolkit-automation <43144436+aws-toolkit-automation@users.noreply.github.com> Co-authored-by: Laxman Reddy <141967714+laileni-aws@users.noreply.github.com> Co-authored-by: Laxman Reddy <141967714+LaxmanReddyAileni@users.noreply.github.com> Co-authored-by: Tai Lai <ctlai95@gmail.com> Co-authored-by: Nikolas Komonen <118216176+nkomonen-amazon@users.noreply.github.com> Co-authored-by: Will Lo <96078566+Will-ShaoHua@users.noreply.github.com> Co-authored-by: Justin M. Keyes <jmkeyes@amazon.com> Co-authored-by: Bryce Ito <bryceito@amazon.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: yi-jin-0 <141669322+yi-jin-0@users.noreply.github.com> Co-authored-by: Zoe Lin <60411978+zixlin7@users.noreply.github.com> Co-authored-by: Santiago Martin <143631912+sannicm@users.noreply.github.com> Co-authored-by: Will Lovo <wlovo@users.noreply.github.com> Co-authored-by: Will Lovo <wlovo@amazon.com> Co-authored-by: Jed A <103950110+JedML@users.noreply.github.com> Co-authored-by: Manthan <manthanpatel.mj@gmail.com> Co-authored-by: Tai Lai <tlaiamz@amazon.com> Co-authored-by: invictus <149003065+ashishrp-aws@users.noreply.github.com>
In AWS Toolkit 2.3.0, |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Problem
Steps to reproduce the issue
Expected behavior
My assumption is that the file scan is just never completing because there are so many files and if so, perhaps you could utilize the user's search exclude list to ignore irrelevant files.
Or give me an option to fully disable this feature and prevent CFN template discovery / watching.
System details (run the
AWS: About Toolkitcommand)The text was updated successfully, but these errors were encountered: