Integration Test Infrastructure #1834
Conversation
TheRealPiotrP
requested review from
DustinCampbell,
WardenGnaw and
gregg-miskelly
TheRealPiotrP
force-pushed
the
dev/piotrp/testInfra
branch
2 times, most recently
from
ca4e802
to
8c3d7fa
Compare
.vscode/launch.json
Outdated
| @@ -7,6 +7,7 @@ | |||
| "request": "launch", | |||
| "runtimeExecutable": "${execPath}", | |||
| "args": [ | |||
| "${workspaceRoot}/test/integrationTests/testAssets/slnWithCsproj", | |||
There was a problem hiding this comment.
I'm still not a super fan of this particular change because it'll make it more of a pain to debug customer repros. I've often had a customer repro project and relied on the fact that subsequent F5 invocations (after the first one) would load the previously loaded workspace. This would require me to change launch.json to get that behavior.
There was a problem hiding this comment.
Ok, I'll revert this to unblock your workflow.
There was a problem hiding this comment.
Thanks! I think it looks good to merge otherwise.
TheRealPiotrP
force-pushed
the
dev/piotrp/testInfra
branch
from
8c3d7fa
to
2c66a69
Compare
|
Out of curiosity, will |
|
I did not merge project.json in this PR. I have it in a private branch if we decide we need it. That said, even if it failed, you can run the integration test projects individually to work around such platform-specific issues! |
Move unit tests into their own folder. Prepare scripts for unit + integration tests.
TheRealPiotrP
force-pushed
the
dev/piotrp/testInfra
branch
from
2c66a69
to
10cbfef
Compare
Don't forget that project.json is already deprecated by the extension. 😄 |
I'm splitting up #1826 into an infra PR and a feature PR. I'd like to merge the infra ASAP. Once this is done, I'll add some integration tests focused on the launch.json/tasks.json interactions. Specifically, I'd like some tests that are able to debug starting at a clean solution to validate the interactions between launch.json and tasks.json. Once this is in-place, I will add back the taskProvider work which will piggy-back on top of the aforementioned tests.
I would like not to squash this PR. I have created meaningful commits which separate the creation of the new infrastructure from the creation of test assets. Folks wishing to add new test assets can use the existing commits as a tempalte.
Test Infrastructure Changes
My strategy in updating the test infrastructure is two-fold. First, I wanted to be able to run integration tests in the context of representative workspaces. Second, I wanted to be able to access data shared with VS Code which is not subsequently queryable via VS Code API's. Here are the key components of the change:
Test Assets
I introduced a new top-level folder called
TestAssets. Within this folder you will find three representative workspaces:These workspaces are accompanied by matching .ts files which describe, from a test expectations perspective, the contents of the workspaces.
omnisharp-vscode launch.json updates [does not affect the extension-produced launch.json]
The launch.json has been updated to have test execution configurations for each of these workspaces. Selecting a given configuration will open the VS Code Test Host targetted at the specified TestAsset workspace. All of the tests running within that Test Host will then be interacting directly with the target workspace, allowing them to produce predictable results while exhibiting their ability to reason about that wokspace type. Note that in order to run through all test scenarios you must iterate through all available Test Configurations.
package.json updates
package.json was updated to allow cmd-line execution of each of the new Test Configurations. The base
npm testcommand was updated to iterate through all three configurations, thereby ensuring that a single cmd-line invocation will execute all tests.