Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jetbrains integration smoke test #8180

Merged
merged 1 commit into from
Mar 2, 2022
Merged

Conversation

iQQBot
Copy link
Contributor

@iQQBot iQQBot commented Feb 13, 2022

Description

This PR is Jetbrains integration smoke test, we use Github Action to run JetBrains Gateway because it has macos and windows machine we need a graphic environment

The integration smoke test step

  1. start a preview environment for main branch
  2. start a new workspace, using special JetBrains IDE
  3. pass oauthToken, ideURL and gatewayLink to Github Action
  4. Github Action will start JetBrains Gateway, and install GatewayPlugin then connect to the workspace
  5. Once it connects, test is success
  6. Notice Github action is always successful, just using this to start a graphic environment and start Jetbrains Gateway, it will timeout ~20min

TO REVIEWERS

  1. This PR is bese on Configure ide integration test to run automatically #7380, please review and merge it first
  2. I use jetbrains-auto-update.yml file to trigger Github action, because if we don't merge to the default branch, Github action doesn't allow manual run workflow_dispatch with a new workflow, if it is ready for merge, I will rename

Related Issue(s)

Relate #7023, fixes #8507

How to test

Open this branch in Gitpod, then run

werft run github -j .werft/run-integration-tests-ide.yaml -a testPattern=jetbrains.test -a version=pd-jetbrains-integration-test.56 -a namespace=staging-pd-jetbrains-integration-test -a username=test-user -a jetbrains-ide=goland

will start an integration test on namespace staging-pd-jetbrains-integration-test, currently gateway plugin and backend plugin is hard code in this branch

When test is done, you can go to Github Action link check the test-video, maybe need sometime to wait timeout

Release Notes

NONE

Documentation

@codecov
Copy link

codecov bot commented Feb 13, 2022

Codecov Report

Merging #8180 (7aacaf3) into main (0c44131) will not change coverage.
The diff coverage is 0.00%.

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #8180   +/-   ##
=======================================
  Coverage   11.17%   11.17%           
=======================================
  Files          18       18           
  Lines         993      993           
=======================================
  Hits          111      111           
  Misses        880      880           
  Partials        2        2           
Flag Coverage Δ
components-gitpod-cli-app 11.17% <0.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
components/gitpod-cli/cmd/credential-helper.go 6.66% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c4ab839...7aacaf3. Read the comment docs.

@iQQBot iQQBot force-pushed the pd/jetbrains-integration-test branch 3 times, most recently from 949a73f to 8819c5c Compare February 13, 2022 19:17
@iQQBot iQQBot closed this Feb 13, 2022
@iQQBot iQQBot force-pushed the pd/jetbrains-integration-test branch from 8819c5c to 854c81c Compare February 13, 2022 19:18
@roboquat roboquat added size/XS and removed size/XXL labels Feb 13, 2022
@iQQBot iQQBot reopened this Feb 14, 2022
@roboquat roboquat added size/XXL and removed size/XS labels Feb 14, 2022
@iQQBot iQQBot force-pushed the pd/jetbrains-integration-test branch 15 times, most recently from 21b2fdd to eca4fb6 Compare February 16, 2022 19:38
@mustard-mh
Copy link
Contributor

@iQQBot
Copy link
Contributor Author

iQQBot commented Feb 28, 2022

image

It only takes two or three seconds, is it really finished?

https://werft.gitpod-dev.com/job/gitpod-run-integration-tests-ide-pd-jetbrains-integrationa.4 takes 7 mins https://werft.gitpod-dev.com/job/gitpod-run-integration-tests-ide-pd-jetbrains-integrationa.5 takes a few seconds

😂 forget push

@iQQBot iQQBot force-pushed the pd/jetbrains-integration-test branch from a62586f to 71dbebb Compare February 28, 2022 09:55
@iQQBot
Copy link
Contributor Author

iQQBot commented Feb 28, 2022

@mustard-mh When test is done, you can go to Github Action link check the test-video, maybe need sometime to wait timeout

also, update on how to test section
and please use 48 version, it add some timeout

Copy link
Contributor

@mustard-mh mustard-mh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How to check if this Github Action Video is generated by the werft job I created

@iQQBot
Copy link
Contributor Author

iQQBot commented Feb 28, 2022

How to check if this Github Action Video is generated by the werft job I created

will add log for github action

@mustard-mh
Copy link
Contributor

If Github Action fails with this 503 loop, werft job fails only after 30m timeout

image

@iQQBot iQQBot force-pushed the jp/run-integration-test branch 2 times, most recently from e0f313f to 07b58c4 Compare March 1, 2022 16:02
Base automatically changed from jp/run-integration-test to main March 1, 2022 16:06
@iQQBot iQQBot force-pushed the pd/jetbrains-integration-test branch 5 times, most recently from c06c5d1 to f74fe9f Compare March 1, 2022 20:29
@iQQBot
Copy link
Contributor Author

iQQBot commented Mar 1, 2022

@mustard-mh I update some code, you can have a try again, But unfortunately, when creating a github action we don't get the action job id from the API, so we can't correspond to it, so we basically have to look it up by time

@iQQBot iQQBot force-pushed the pd/jetbrains-integration-test branch 2 times, most recently from f14f1ff to 0dc2d4e Compare March 2, 2022 14:25
@mustard-mh mustard-mh marked this pull request as draft March 2, 2022 14:37
@iQQBot iQQBot force-pushed the pd/jetbrains-integration-test branch from 0dc2d4e to 7aacaf3 Compare March 2, 2022 16:13
@mustard-mh mustard-mh marked this pull request as ready for review March 2, 2022 16:43
Copy link
Contributor

@mustard-mh mustard-mh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@roboquat roboquat merged commit 7ac5d9d into main Mar 2, 2022
@roboquat roboquat deleted the pd/jetbrains-integration-test branch March 2, 2022 16:44
@roboquat roboquat added deployed: IDE IDE change is running in production deployed Change is completely running in production labels Mar 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
deployed: IDE IDE change is running in production deployed Change is completely running in production release-note-none size/XXL team: IDE
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[jetbrains] integration testing cut-through
3 participants