Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(amazonq): generateZipTestGen is unreliable #6290

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

ctlai95
Copy link
Contributor

@ctlai95 ctlai95 commented Dec 24, 2024

Problem

Flaky test generateZipTestGen sometimes fails with

  1) zipUtil
       generateZipTestGen
         Should generate zip for test generation successfully:
     Error: done() called multiple times in test <zipUtil generateZipTestGen Should generate zip for test generation successfully> of file /Users/runner/work/aws-toolkit-vscode/aws-toolkit-vscode/packages/amazonq/dist/test/unit/codewhisperer/util/zipUtil.test.js; in addition, done() received error: EntryNotFound (FileSystemError): Error: ENOENT: no such file or directory, scandir '/test/zip/utgRequiredArtifactsDir'
    at Function.e (/private/tmp/.vscode-test/vscode-darwin-arm64-1.83.0/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:127:26740)
    at Object.readDirectory (/private/tmp/.vscode-test/vscode-darwin-arm64-1.83.0/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:127:24744)
    at async processDirectory (/Users/runner/work/aws-toolkit-vscode/aws-toolkit-vscode/packages/core/src/codewhisperer/util/zipUtil.ts:190:29)
    at async ZipUtil.processMetadataDir (/Users/runner/work/aws-toolkit-vscode/aws-toolkit-vscode/packages/core/src/codewhisperer/util/zipUtil.ts:212:9)
    at async ZipUtil.zipProject (/Users/runner/work/aws-toolkit-vscode/aws-toolkit-vscode/packages/core/src/codewhisperer/util/zipUtil.ts:230:13)
    at async ZipUtil.generateZipTestGen (/Users/runner/work/aws-toolkit-vscode/aws-toolkit-vscode/packages/core/src/codewhisperer/util/zipUtil.ts:602:41)
    at async Context.<anonymous> (/Users/runner/work/aws-toolkit-vscode/aws-toolkit-vscode/packages/amazonq/test/unit/codewhisperer/util/zipUtil.test.ts:159:28) {
  code: 'FileNotFound'

Issue: #6160

Solution

Based on the error, the test is trying to readdir on a mock dirpath even though the entire zipProject method should be stubbed. This can be confirmed by commenting out the following line:

sinon.stub(zipUtil as any, 'zipProject').resolves(zipFilePath)

which will give the same error

Error: ENOENT: no such file or directory, scandir '/test/zip/utgRequiredArtifactsDir'

So instead of stubbing which seems to be unreliable, we can let the test actually create the zip file.


  • Treat all work as PUBLIC. Private feature/x branches will not be squash-merged at release time.
  • Your code changes must meet the guidelines in CONTRIBUTING.md.
  • License: I confirm that my contribution is made under the terms of the Apache 2.0 license.

Copy link

  • This pull request modifies code in src/* but no tests were added/updated.
    • Confirm whether tests should be added or ensure the PR description explains why tests are not required.

@ctlai95 ctlai95 changed the title wip test: wip Jan 2, 2025
@ctlai95 ctlai95 changed the title test: wip test(amazonq): wip Jan 2, 2025
@ctlai95 ctlai95 closed this Jan 2, 2025
@ctlai95 ctlai95 reopened this Jan 2, 2025
@ctlai95 ctlai95 changed the title test(amazonq): wip test(amazonq): generateZipTestGen is unreliable Jan 2, 2025
@ctlai95 ctlai95 marked this pull request as ready for review January 2, 2025 23:14
@ctlai95 ctlai95 requested a review from a team as a code owner January 2, 2025 23:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant