Implementation of testing api for testing vscode extensions language features [Part 2] #802

svor · 2020-07-10T17:57:06Z

What does this PR do?

This PR is based on #597 and was created to continue work on the task about providing language's TestAPI.

It introduces testing API that allows you to talk to a specific vscode extensions language features. It gives the ability to create tests that call into a vscode language extension for two reasons:

Tests can be run against newer version of language plugins so we know if a language plugin can be updated or something needs to be fixed
Che tests can talk directly with the language server so we know that if the language server request fails then it's an actual failure, rather than a test failing for a different reason and making it look like it was the cause
Examples of such tests can be found in: https://github.com/che-incubator/Che-Java-Tests

What issues does this PR fix or reference?

eclipse-che/che#14409
partially eclipse-che/che#15797

Signed-off-by: Josh Pinkney <joshpinkney@gmail.com>

Signed-off-by: svor <vsvydenk@redhat.com>

che-bot · 2020-07-10T18:08:32Z

❌ E2E Happy path tests failed ❗

See Details

name	link
che-theia	docker.io/maxura/che-theia:802
che-theia-endpoint-runtime-binary	docker.io/maxura/che-theia-endpoint-runtime-binary:802

Tested with Eclipse Che Single User on K8S (minikube v1.1.1)

ℹ️ Use comment "crw-ci-test" to rerun happy path E2E test.

svor · 2020-07-27T12:32:26Z

crw-ci-test

che-bot · 2020-07-27T12:42:21Z

❌ E2E Happy path tests failed ❗

See Details

name	link
che-theia	docker.io/maxura/che-theia:802
che-theia-endpoint-runtime-binary	docker.io/maxura/che-theia-endpoint-runtime-binary:802

Tested with Eclipse Che Single User on K8S (minikube v1.1.1)

ℹ️ Use comment "crw-ci-test" to rerun happy path E2E test.

benoitf

still requiring changes:

move it to '@eclipse-che/plugin' , languages namespace or tests
cleanup dependencies and dev-dependencies
re-use most of declarations from existing vscode/theia .d.ts
it is not enabled on any che-theia assembly.

Signed-off-by: svor <vsvydenk@redhat.com>

che-bot · 2020-07-30T08:20:59Z

❌ E2E Happy path tests failed ❗

See Details

name	link
che-theia	docker.io/maxura/che-theia:802
che-theia-endpoint-runtime-binary	docker.io/maxura/che-theia-endpoint-runtime-binary:802

Tested with Eclipse Che Single User on K8S (minikube v1.1.1)

ℹ️ Use comment "crw-ci-test" to rerun happy path E2E test.

che-bot · 2020-08-03T13:19:41Z

❌ E2E Happy path tests failed ❗

See Details

name	link
che-theia	docker.io/maxura/che-theia:802
che-theia-endpoint-runtime-binary	docker.io/maxura/che-theia-endpoint-runtime-binary:802

Tested with Eclipse Che Single User on K8S (minikube v1.1.1)

ℹ️ Use comment "crw-ci-test" to rerun happy path E2E test.

Signed-off-by: svor <vsvydenk@redhat.com>

che-bot · 2020-08-12T09:08:12Z

✅ E2E Happy path tests succeed 🎉

See Details

name	link
che-theia	docker.io/maxura/che-theia:802
che-theia-endpoint-runtime-binary	docker.io/maxura/che-theia-endpoint-runtime-binary:802

Tested with Eclipse Che Single User on K8S (minikube v1.1.1)

extensions/eclipse-che-theia-plugin-ext/src/browser/che-languages-test-api.ts

extensions/eclipse-che-theia-plugin/src/che-proposed.d.ts

extensions/eclipse-che-theia-plugin-ext/src/browser/che-languages-test-api.ts

…nents Signed-off-by: svor <vsvydenk@redhat.com>

che-bot · 2020-08-12T13:03:10Z

✅ E2E Happy path tests succeed 🎉

See Details

name	link
che-theia	docker.io/maxura/che-theia:802
che-theia-endpoint-runtime-binary	docker.io/maxura/che-theia-endpoint-runtime-binary:802

Tested with Eclipse Che Single User on K8S (minikube v1.1.1)

Signed-off-by: svor <vsvydenk@redhat.com>

svor · 2020-08-20T08:41:25Z

@benoitf PR was updated to re-use declarations from theia.d.ts, could you please take another look

che-bot · 2020-08-20T09:00:59Z

✅ E2E Happy path tests succeed 🎉

See Details

name	link
che-theia	docker.io/maxura/che-theia:802
che-theia-endpoint-runtime-binary	docker.io/maxura/che-theia-endpoint-runtime-binary:802

Tested with Eclipse Che Single User on K8S (minikube v1.1.1)

lot of changes have been done, cancelling my request for changes

benoitf · 2020-08-20T09:44:19Z

@svor : do you have a testing plug-in that you're using ?

like a given branch of https://github.com/che-incubator/Che-Java-Tests or I can use master branch from there ?

svor · 2020-08-20T11:18:57Z

@benoitf test plugin is not ready yet, i'm working on adapting it

svor · 2020-08-20T14:26:32Z

@benoitf
Branch with adapted tests is https://github.com/svor/che-java-tests/tree/sv/tests but it is still in progress
here is a video about how it looks when running tests inside che-the: https://youtu.be/vT0eSgexYB0

Signed-off-by: svor <vsvydenk@redhat.com>

svor · 2020-08-21T12:46:15Z

To run tests from test plugin use this devfile:

apiVersion: 1.0.0
metadata:
  name: che-java-sample-tests
components:
  - id: redhat/java11/latest
    type: chePlugin
  - reference: https://raw.githubusercontent.com/svor/che-java-tests/sv/tests/meta.yaml
    type: chePlugin
  - reference: https://raw.githubusercontent.com/svor/che-plugin-registry/sv/che-theia2/v3/plugins/eclipse/che-theia/next/meta.yaml
    memoryLimit: 1024Mi
    type: cheEditor

che-bot · 2020-08-21T13:21:33Z

✅ E2E Happy path tests succeed 🎉

See Details

name	link
che-theia	docker.io/maxura/che-theia:802
che-theia-endpoint-runtime-binary	docker.io/maxura/che-theia-endpoint-runtime-binary:802

Tested with Eclipse Che Single User on K8S (minikube v1.1.1)

benoitf

oups, it seems my review was not published
clicking submit now

extensions/eclipse-che-theia-plugin-ext/src/browser/type-converters.ts

Signed-off-by: svor <vsvydenk@redhat.com>

che-bot · 2020-09-08T09:00:10Z

✅ E2E Happy path tests succeed 🎉

See Details

name	link
che-theia	docker.io/maxura/che-theia:802
che-theia-endpoint-runtime-binary	docker.io/maxura/che-theia-endpoint-runtime-binary:802

Tested with Eclipse Che Single User on K8S (minikube v1.1.1)

benoitf

tested on a custom che instance

…features [Part 2] (eclipse-che#802) * Added extension testing Signed-off-by: Josh Pinkney <joshpinkney@gmail.com> * Fix build errors Signed-off-by: svor <vsvydenk@redhat.com> * Add empty line Signed-off-by: svor <vsvydenk@redhat.com> * Add selectionRange language action Signed-off-by: svor <vsvydenk@redhat.com> * Move language test API under che namespace Signed-off-by: svor <vsvydenk@redhat.com> * Add empty line Signed-off-by: svor <vsvydenk@redhat.com> * Add empty line Signed-off-by: svor <vsvydenk@redhat.com> * Add empty line Signed-off-by: svor <vsvydenk@redhat.com> * use thei.Uri instead of plugin-ext/lib/common/uri-components.UriComponents Signed-off-by: svor <vsvydenk@redhat.com> * Update language test api to use types from theia/plugin Signed-off-by: svor <vsvydenk@redhat.com> * Fix declaration request Signed-off-by: svor <vsvydenk@redhat.com> * code cleanup Signed-off-by: svor <vsvydenk@redhat.com> Co-authored-by: Josh Pinkney <joshpinkney@gmail.com>

JPinkney and others added 5 commits February 18, 2020 18:32

Added extension testing

d992ae8

Signed-off-by: Josh Pinkney <joshpinkney@gmail.com>

merge with master

7c6e36f

Signed-off-by: svor <vsvydenk@redhat.com>

Fix build errors

5528984

Signed-off-by: svor <vsvydenk@redhat.com>

Add empty line

f2f47f6

Signed-off-by: svor <vsvydenk@redhat.com>

Add selectionRange language action

7d4b45c

Signed-off-by: svor <vsvydenk@redhat.com>

svor requested review from benoitf, ericwill and tsmaeder July 10, 2020 17:57

svor requested review from azatsarynnyy, RomanNikitenko and vzhukovs as code owners July 10, 2020 17:57

svor self-assigned this Jul 10, 2020

ericwill mentioned this pull request Jul 10, 2020

Implementation of testing api for testing vscode extensions language features #597

Closed

benoitf previously requested changes Jul 27, 2020

View reviewed changes

merge with master

ac7bce7

Signed-off-by: svor <vsvydenk@redhat.com>

Merge branch 'master' into testing-ext2

dbb80f4

svor added 2 commits August 3, 2020 19:58

Merge branch 'master' into testing-ext2

f1762b7

Move language test API under che namespace

9e4a4b2

Signed-off-by: svor <vsvydenk@redhat.com>

svor requested review from vinokurig and vitaliy-guliy as code owners August 12, 2020 08:22

svor added 3 commits August 12, 2020 11:26

Add empty line

c3bdcf6

Signed-off-by: svor <vsvydenk@redhat.com>

Add empty line

e878e91

Signed-off-by: svor <vsvydenk@redhat.com>

Add empty line

98e6a84

Signed-off-by: svor <vsvydenk@redhat.com>

benoitf reviewed Aug 12, 2020

View reviewed changes

extensions/eclipse-che-theia-plugin-ext/src/browser/che-languages-test-api.ts Outdated Show resolved Hide resolved

benoitf reviewed Aug 12, 2020

View reviewed changes

extensions/eclipse-che-theia-plugin/src/che-proposed.d.ts Outdated Show resolved Hide resolved

benoitf reviewed Aug 12, 2020

View reviewed changes

extensions/eclipse-che-theia-plugin/src/che-proposed.d.ts Outdated Show resolved Hide resolved

benoitf reviewed Aug 12, 2020

View reviewed changes

extensions/eclipse-che-theia-plugin-ext/src/browser/che-languages-test-api.ts Outdated Show resolved Hide resolved

use thei.Uri instead of plugin-ext/lib/common/uri-components.UriCompo…

6f8224f

…nents Signed-off-by: svor <vsvydenk@redhat.com>

Update language test api to use types from theia/plugin

f40aeb6

Signed-off-by: svor <vsvydenk@redhat.com>

Fix declaration request

7b62137

Signed-off-by: svor <vsvydenk@redhat.com>

svor mentioned this pull request Sep 2, 2020

Adapt plugin to use new language test API che-incubator/che-java-tests#5

Merged

benoitf reviewed Sep 8, 2020

View reviewed changes

extensions/eclipse-che-theia-plugin-ext/src/browser/type-converters.ts Outdated Show resolved Hide resolved

extensions/eclipse-che-theia-plugin-ext/src/browser/type-converters.ts Outdated Show resolved Hide resolved

code cleanup

a636459

Signed-off-by: svor <vsvydenk@redhat.com>

benoitf approved these changes Sep 9, 2020

View reviewed changes

svor merged commit 468b008 into eclipse-che:master Sep 10, 2020

che-bot added this to the 7.19 milestone Sep 10, 2020

vinokurig pushed a commit that referenced this pull request Apr 6, 2021

chore(devfile): use the new plugin id for the github pr plugin (#802)

3f08ae7

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implementation of testing api for testing vscode extensions language features [Part 2] #802

Implementation of testing api for testing vscode extensions language features [Part 2] #802

svor commented Jul 10, 2020

che-bot commented Jul 10, 2020

svor commented Jul 27, 2020

che-bot commented Jul 27, 2020

benoitf left a comment

che-bot commented Jul 30, 2020

che-bot commented Aug 3, 2020

che-bot commented Aug 12, 2020

che-bot commented Aug 12, 2020

svor commented Aug 20, 2020

che-bot commented Aug 20, 2020

benoitf commented Aug 20, 2020 •

edited

Loading

svor commented Aug 20, 2020

svor commented Aug 20, 2020

svor commented Aug 21, 2020

che-bot commented Aug 21, 2020

benoitf left a comment

che-bot commented Sep 8, 2020

benoitf left a comment

Implementation of testing api for testing vscode extensions language features [Part 2] #802

Implementation of testing api for testing vscode extensions language features [Part 2] #802

Conversation

svor commented Jul 10, 2020

What does this PR do?

What issues does this PR fix or reference?

che-bot commented Jul 10, 2020

svor commented Jul 27, 2020

che-bot commented Jul 27, 2020

benoitf left a comment

Choose a reason for hiding this comment

che-bot commented Jul 30, 2020

che-bot commented Aug 3, 2020

che-bot commented Aug 12, 2020

che-bot commented Aug 12, 2020

svor commented Aug 20, 2020

che-bot commented Aug 20, 2020

benoitf commented Aug 20, 2020 • edited Loading

svor commented Aug 20, 2020

svor commented Aug 20, 2020

svor commented Aug 21, 2020

che-bot commented Aug 21, 2020

benoitf left a comment

Choose a reason for hiding this comment

che-bot commented Sep 8, 2020

benoitf left a comment

Choose a reason for hiding this comment

benoitf commented Aug 20, 2020 •

edited

Loading