-
Notifications
You must be signed in to change notification settings - Fork 4.1k
feat: run e2e tests on vsc release #7190
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,125 @@ | ||
| name: 'E2E Tests' | ||
|
|
||
| on: | ||
| workflow_call: | ||
| inputs: | ||
| is_fork: | ||
| description: 'Whether this is running on a fork' | ||
| type: boolean | ||
| default: false | ||
| secrets: | ||
| CI_GITHUB_TOKEN: | ||
| required: true | ||
| GITHUB_TOKEN: | ||
| required: true | ||
| GH_ACTIONS_SSH_TEST_KEY_PEM: | ||
| required: false | ||
| GH_ACTIONS_SSH_TEST_DNS_NAME: | ||
| required: false | ||
|
|
||
| jobs: | ||
| vscode-get-test-file-matrix: | ||
| runs-on: ubuntu-latest | ||
| outputs: | ||
| test_file_matrix: ${{ steps.vscode-get-test-file-matrix.outputs.test_file_matrix }} | ||
| steps: | ||
| - uses: actions/checkout@v5 | ||
| - uses: actions/setup-node@v4 | ||
| with: | ||
| node-version-file: ".nvmrc" | ||
|
|
||
| - name: Cache npm | ||
| uses: actions/cache@v4 | ||
| with: | ||
| path: ~/.npm | ||
| key: ${{ runner.os }}-npm-cache-matrix-${{ hashFiles('core/package-lock.json', 'extensions/vscode/package-lock.json') }} | ||
|
|
||
| - name: Cache packages node_modules | ||
| uses: actions/cache@v4 | ||
| with: | ||
| path: | | ||
| packages/*/node_modules | ||
| key: ${{ runner.os }}-packages-node-modules-${{ hashFiles('packages/*/package-lock.json') }} | ||
|
|
||
| - name: Cache core node modules | ||
| uses: actions/cache@v4 | ||
| with: | ||
| path: core/node_modules | ||
| key: ${{ runner.os }}-core-node-modules-${{ hashFiles('core/package-lock.json') }} | ||
|
|
||
| - name: Cache vscode extension node modules | ||
| uses: actions/cache@v4 | ||
| id: vscode-cache | ||
| with: | ||
| path: extensions/vscode/node_modules | ||
| key: ${{ runner.os }}-vscode-node-modules-${{ hashFiles('extensions/vscode/package-lock.json') }} | ||
|
|
||
| - name: Build packages and get test files | ||
| id: vscode-get-test-file-matrix | ||
| run: | | ||
| node ./scripts/build-packages.js | ||
| cd extensions/vscode | ||
| npm ci | ||
| npm run e2e:compile | ||
| if [[ "${{ inputs.is_fork }}" == "true" ]]; then | ||
| # Exclude SSH tests for forks | ||
| FILES=$(ls -1 e2e/_output/tests/*.test.js | grep -v "SSH" | jq -R . | jq -s .) | ||
| else | ||
| # Include all tests for non-forks | ||
| FILES=$(ls -1 e2e/_output/tests/*.test.js | jq -R . | jq -s .) | ||
| fi | ||
| echo "test_file_matrix<<EOF" >> $GITHUB_OUTPUT | ||
| echo "$FILES" >> $GITHUB_OUTPUT | ||
| echo "EOF" >> $GITHUB_OUTPUT | ||
| env: | ||
| # https://github.com/microsoft/vscode-ripgrep/issues/9#issuecomment-643965333 | ||
| GITHUB_TOKEN: ${{ secrets.CI_GITHUB_TOKEN }} | ||
|
|
||
| - name: Debug Outputs | ||
| run: | | ||
| echo "Test files: ${{ steps.vscode-get-test-file-matrix.outputs.test_file_matrix }}" | ||
|
|
||
| vscode-package-extension-linux: | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 15 | ||
| steps: | ||
| - uses: actions/checkout@v5 | ||
| with: | ||
| fetch-depth: 0 | ||
|
|
||
| - name: Build VS Code extension | ||
| uses: ./.github/actions/build-vscode-extension | ||
| with: | ||
| platform: linux | ||
| arch: x64 | ||
| npm_config_arch: x64 | ||
| pre-release: false | ||
| commit-sha: ${{ github.sha }} | ||
| github-token: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
| - name: Upload build artifact | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: vscode-extension-build-Linux | ||
| path: extensions/vscode/build | ||
|
|
||
| vscode-e2e-tests: | ||
| name: ${{ matrix.test_file || 'unknown' }} (${{ matrix.command }}) | ||
| needs: [vscode-get-test-file-matrix, vscode-package-extension-linux] | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 15 | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| test_file: ${{ fromJson(needs.vscode-get-test-file-matrix.outputs.test_file_matrix) }} | ||
| command: ["e2e:ci:run", "e2e:ci:run-yaml"] | ||
| steps: | ||
| - uses: actions/checkout@v5 | ||
| - name: Run VS Code E2E test | ||
| uses: ./.github/actions/run-vscode-e2e-test | ||
| with: | ||
| test_file: ${{ matrix.test_file }} | ||
| command: ${{ matrix.command }} | ||
| ssh_key: ${{ secrets.GH_ACTIONS_SSH_TEST_KEY_PEM }} | ||
| ssh_host: ${{ secrets.GH_ACTIONS_SSH_TEST_DNS_NAME }} | ||
| is_fork: ${{ inputs.is_fork }} | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Missing newline at end of file. Add a newline character to follow YAML best practices. |
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -110,6 +110,7 @@ jobs: | |
| runs-on: ubuntu-latest | ||
| needs: | ||
| - build | ||
| - vscode-e2e-tests | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@v5 | ||
|
|
@@ -135,10 +136,23 @@ jobs: | |
| token: ${{ secrets.CI_GITHUB_TOKEN }} | ||
| repository: continuedev/continue | ||
|
|
||
| vscode-e2e-tests: | ||
| needs: check_release_name | ||
| if: needs.check_release_name.outputs.should_run == 'true' || github.event_name == 'workflow_dispatch' | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Consider adding 'if: always()' or 'if: success()' to clarify when E2E tests should run in the main workflow. |
||
| uses: ./.github/workflows/e2e-tests.yml | ||
| with: | ||
| is_fork: false | ||
| secrets: | ||
| CI_GITHUB_TOKEN: ${{ secrets.CI_GITHUB_TOKEN }} | ||
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| GH_ACTIONS_SSH_TEST_KEY_PEM: ${{ secrets.GH_ACTIONS_SSH_TEST_KEY_PEM }} | ||
| GH_ACTIONS_SSH_TEST_DNS_NAME: ${{ secrets.GH_ACTIONS_SSH_TEST_DNS_NAME }} | ||
|
|
||
| publish: | ||
| runs-on: ubuntu-latest | ||
| needs: | ||
| - build | ||
| - vscode-e2e-tests | ||
| if: github.event_name != 'workflow_dispatch' || github.event.inputs.publish_build == 'true' | ||
| permissions: | ||
| contents: write | ||
|
|
||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -192,112 +192,15 @@ jobs: | |||||
| AZURE_OPENAI_GPT41_API_KEY: ${{ secrets.AZURE_OPENAI_GPT41_API_KEY }} | ||||||
| VOYAGE_API_KEY: ${{ secrets.VOYAGE_API_KEY }} | ||||||
|
|
||||||
| vscode-get-test-file-matrix: | ||||||
| runs-on: ubuntu-latest | ||||||
| outputs: | ||||||
| test_file_matrix: ${{ steps.vscode-get-test-file-matrix.outputs.test_file_matrix }} | ||||||
| steps: | ||||||
| - uses: actions/checkout@v5 | ||||||
| - uses: actions/setup-node@v4 | ||||||
| with: | ||||||
| node-version-file: ".nvmrc" | ||||||
|
|
||||||
| - name: Cache npm | ||||||
| uses: actions/cache@v4 | ||||||
| with: | ||||||
| path: ~/.npm | ||||||
| key: ${{ runner.os }}-npm-cache-matrix-${{ hashFiles('core/package-lock.json', 'extensions/vscode/package-lock.json') }} | ||||||
|
|
||||||
| - name: Cache packages node_modules | ||||||
| uses: actions/cache@v4 | ||||||
| with: | ||||||
| path: | | ||||||
| packages/*/node_modules | ||||||
| key: ${{ runner.os }}-packages-node-modules-${{ hashFiles('packages/*/package-lock.json') }} | ||||||
|
|
||||||
| - name: Cache core node modules | ||||||
| uses: actions/cache@v4 | ||||||
| with: | ||||||
| path: core/node_modules | ||||||
| key: ${{ runner.os }}-core-node-modules-${{ hashFiles('core/package-lock.json') }} | ||||||
|
|
||||||
| - name: Cache vscode extension node modules | ||||||
| uses: actions/cache@v4 | ||||||
| id: vscode-cache | ||||||
| with: | ||||||
| path: extensions/vscode/node_modules | ||||||
| key: ${{ runner.os }}-vscode-node-modules-${{ hashFiles('extensions/vscode/package-lock.json') }} | ||||||
|
|
||||||
| - name: Build packages and get test files | ||||||
| id: vscode-get-test-file-matrix | ||||||
| run: | | ||||||
| node ./scripts/build-packages.js | ||||||
| cd extensions/vscode | ||||||
| npm ci | ||||||
| npm run e2e:compile | ||||||
| if [[ "${{ github.event.pull_request.head.repo.fork }}" == "true" || "${{ github.actor }}" == "dependabot[bot]" ]]; then | ||||||
| # Exclude SSH tests for forks | ||||||
| FILES=$(ls -1 e2e/_output/tests/*.test.js | grep -v "SSH" | jq -R . | jq -s .) | ||||||
| else | ||||||
| # Include all tests for non-forks | ||||||
| FILES=$(ls -1 e2e/_output/tests/*.test.js | jq -R . | jq -s .) | ||||||
| fi | ||||||
| echo "test_file_matrix<<EOF" >> $GITHUB_OUTPUT | ||||||
| echo "$FILES" >> $GITHUB_OUTPUT | ||||||
| echo "EOF" >> $GITHUB_OUTPUT | ||||||
| env: | ||||||
| # https://github.com/microsoft/vscode-ripgrep/issues/9#issuecomment-643965333 | ||||||
| GITHUB_TOKEN: ${{ secrets.CI_GITHUB_TOKEN }} | ||||||
|
|
||||||
| - name: Debug Outputs | ||||||
| run: | | ||||||
| echo "Test files: ${{ steps.vscode-get-test-file-matrix.outputs.test_file_matrix }}" | ||||||
|
|
||||||
| vscode-package-extension-linux: | ||||||
| runs-on: ubuntu-latest | ||||||
| timeout-minutes: 15 | ||||||
| steps: | ||||||
| - uses: actions/checkout@v5 | ||||||
| with: | ||||||
| fetch-depth: 0 | ||||||
|
|
||||||
| - name: Build VS Code extension | ||||||
| uses: ./.github/actions/build-vscode-extension | ||||||
| with: | ||||||
| platform: linux | ||||||
| arch: x64 | ||||||
| npm_config_arch: x64 | ||||||
| pre-release: false | ||||||
| commit-sha: ${{ github.sha }} | ||||||
| github-token: ${{ secrets.GITHUB_TOKEN }} | ||||||
|
|
||||||
| - name: Upload build artifact | ||||||
| uses: actions/upload-artifact@v4 | ||||||
| with: | ||||||
| name: vscode-extension-build-Linux | ||||||
| path: extensions/vscode/build | ||||||
|
|
||||||
| vscode-e2e-tests: | ||||||
| name: ${{ matrix.test_file || 'unknown' }} (${{ matrix.command }}) | ||||||
| needs: [vscode-get-test-file-matrix, vscode-package-extension-linux] | ||||||
| runs-on: ubuntu-latest | ||||||
| timeout-minutes: 15 | ||||||
| # Tests requiring secrets need approval from maintainers | ||||||
| strategy: | ||||||
| fail-fast: false | ||||||
| matrix: | ||||||
| test_file: ${{ fromJson(needs.vscode-get-test-file-matrix.outputs.test_file_matrix) }} | ||||||
| command: ["e2e:ci:run", "e2e:ci:run-yaml"] | ||||||
| steps: | ||||||
| - uses: actions/checkout@v5 | ||||||
| - name: Run VS Code E2E test | ||||||
| uses: ./.github/actions/run-vscode-e2e-test | ||||||
| with: | ||||||
| test_file: ${{ matrix.test_file }} | ||||||
| command: ${{ matrix.command }} | ||||||
| ssh_key: ${{ secrets.GH_ACTIONS_SSH_TEST_KEY_PEM }} | ||||||
| ssh_host: ${{ secrets.GH_ACTIONS_SSH_TEST_DNS_NAME }} | ||||||
| is_fork: ${{ github.event.pull_request.head.repo.fork == true || github.actor == 'dependabot[bot]' }} | ||||||
| uses: ./.github/workflows/e2e-tests.yml | ||||||
| with: | ||||||
| is_fork: ${{ github.event.pull_request.head.repo.fork == true || github.actor == 'dependabot[bot]' }} | ||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Good refactoring! Consider adding a comment explaining that SSH tests are excluded for forks in the reusable workflow.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If this workflow runs on a push event (where the pull_request payload is absent), the expression throws because github.event.pull_request is undefined. Guard the access with a check on github.event_name first to avoid runtime evaluation errors. Prompt for AI agents
Suggested change
|
||||||
| secrets: | ||||||
| CI_GITHUB_TOKEN: ${{ secrets.CI_GITHUB_TOKEN }} | ||||||
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||||||
| GH_ACTIONS_SSH_TEST_KEY_PEM: ${{ secrets.GH_ACTIONS_SSH_TEST_KEY_PEM }} | ||||||
| GH_ACTIONS_SSH_TEST_DNS_NAME: ${{ secrets.GH_ACTIONS_SSH_TEST_DNS_NAME }} | ||||||
|
|
||||||
| jetbrains-tests: | ||||||
| runs-on: ubuntu-latest | ||||||
|
|
@@ -350,8 +253,6 @@ jobs: | |||||
| - vscode-checks | ||||||
| - get-packages-matrix | ||||||
| - packages-checks | ||||||
| - vscode-get-test-file-matrix | ||||||
| - vscode-package-extension-linux | ||||||
| - vscode-e2e-tests | ||||||
| - jetbrains-tests | ||||||
| - pr-review-action-tests | ||||||
|
|
||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The fallback 'unknown' for matrix.test_file suggests potential issues. Consider adding validation to ensure test_file_matrix is properly populated.