Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Display test results in CodeLens #24

Merged
merged 13 commits into from
Nov 26, 2017
3 changes: 2 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,7 @@
"vscode": "^1.0.0"
},
"dependencies": {
"applicationinsights": "^0.21.0"
"applicationinsights": "^0.21.0",
"xmldom": "^0.1.27"
}
}
27 changes: 22 additions & 5 deletions src/dotnetTestExplorer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import { TreeDataProvider, TreeItem, TreeItemCollapsibleState } from "vscode";
import { AppInsightsClient } from "./appInsightsClient";
import { Executor } from "./executor";
import { TestNode } from "./testNode";
import { TestResultsFile } from "./testResultsFile";
import { Utility } from "./utility";

export class DotnetTestExplorer implements TreeDataProvider<TestNode> {
Expand All @@ -16,7 +17,7 @@ export class DotnetTestExplorer implements TreeDataProvider<TestNode> {
*/
private testDirectoryPath: string;

constructor(private context: vscode.ExtensionContext) {}
constructor(private context: vscode.ExtensionContext, private resultsFile: TestResultsFile) { }

/**
* @description
Expand All @@ -41,7 +42,7 @@ export class DotnetTestExplorer implements TreeDataProvider<TestNode> {
*/
public runAllTests(): void {
this.evaluateTestDirectory();
Executor.runInTerminal(`dotnet test${this.checkBuildOption()}${this.checkRestoreOption()}`, this.testDirectoryPath);
Executor.runInTerminal(`dotnet test${this.getDotNetTestOptions()}${this.outputTestResults()}`, this.testDirectoryPath);
AppInsightsClient.sendEvent("runAllTests");
}

Expand All @@ -53,7 +54,7 @@ export class DotnetTestExplorer implements TreeDataProvider<TestNode> {
* to do a restore, so it can be very slow.
*/
public runTest(test: TestNode): void {
Executor.runInTerminal(`dotnet test${this.checkBuildOption()}${this.checkRestoreOption()} --filter FullyQualifiedName~${test.fullName}`, this.testDirectoryPath);
Executor.runInTerminal(`dotnet test${this.getDotNetTestOptions()}${this.outputTestResults()} --filter FullyQualifiedName~${test.fullName}`, this.testDirectoryPath);
AppInsightsClient.sendEvent("runTest");
}

Expand Down Expand Up @@ -175,6 +176,22 @@ export class DotnetTestExplorer implements TreeDataProvider<TestNode> {
return option ? "" : " --no-restore";
}

/**
* @description
* Gets the options for build/restore before running tests.
*/
private getDotNetTestOptions(): string {
return this.checkBuildOption() + this.checkRestoreOption();
}

/**
* @description
* Gets the dotnet test argument to speicfy the output for the test results.
*/
private outputTestResults(): string {
return " --logger 'trx;LogFileName=" + this.resultsFile.fileName + "'";
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I test it on Windows, it is failed, After I change single quote to double quote, it is working. How about you?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It did work for me, but I've replaced with double quotes and it still works for me so hopefully that should fix it for you

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess you are using PowerShell. Both are working on PowerShell, while single quote does not work on CMD.

}

/**
* @description
* Checks to see if the options specify a directory to run the
Expand All @@ -198,7 +215,7 @@ export class DotnetTestExplorer implements TreeDataProvider<TestNode> {
return new Promise((c, e) => {
try {
const results = Executor
.execSync(`dotnet test -t -v=q${this.checkBuildOption()}${this.checkRestoreOption()}`, this.testDirectoryPath)
.execSync(`dotnet test -t -v=q${this.getDotNetTestOptions()}`, this.testDirectoryPath)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You did not add ${this.outputTestResults()}, so the test result will not be shown when VS Code starts. Is it by design?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, please ignore. It only lists the tests.

.split(/[\r\n]+/g)
/*
* The dotnet-cli prefixes all discovered unit tests
Expand All @@ -207,7 +224,7 @@ export class DotnetTestExplorer implements TreeDataProvider<TestNode> {
* structures.
**/
.filter((item) => item && item.startsWith(" "))
.sort((a, b) => a > b ? 1 : b > a ? - 1 : 0 )
.sort((a, b) => a > b ? 1 : b > a ? - 1 : 0)
.map((item) => item.trim());

c(results);
Expand Down
14 changes: 13 additions & 1 deletion src/extension.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,24 @@ import { AppInsightsClient } from "./appInsightsClient";
import { DotnetTestExplorer } from "./dotnetTestExplorer";
import { Executor } from "./executor";
import { TestNode } from "./testNode";
import { TestResultsFile } from "./testResultsFile";
import { TestStatusCodeLensProvider } from "./testStatusCodeLensProvider";
import { Utility } from "./utility";

export function activate(context: vscode.ExtensionContext) {
const dotnetTestExplorer = new DotnetTestExplorer(context);
const testResults = new TestResultsFile();
context.subscriptions.push(testResults);

const dotnetTestExplorer = new DotnetTestExplorer(context, testResults);
vscode.window.registerTreeDataProvider("dotnetTestExplorer", dotnetTestExplorer);
AppInsightsClient.sendEvent("loadExtension");

const codeLensProvider = new TestStatusCodeLensProvider(testResults);
context.subscriptions.push(codeLensProvider);
context.subscriptions.push(vscode.languages.registerCodeLensProvider(
{ language: "csharp", scheme: "file" },
codeLensProvider));

context.subscriptions.push(vscode.commands.registerCommand("dotnet-test-explorer.refreshTestExplorer", () => {
dotnetTestExplorer.refreshTestExplorer();
}));
Expand Down
12 changes: 12 additions & 0 deletions src/testResult.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
export class TestResult {
public constructor(private _name: string, private _outcome: string) {
}

public get testName(): string {
return this._name;
}

public get outcome(): string {
return this._outcome;
}
}
74 changes: 74 additions & 0 deletions src/testResultsFile.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
"use strict";
import * as fs from "fs";
import * as path from "path";
import { setTimeout } from "timers";
import { Disposable, Event, EventEmitter } from "vscode";
import { DOMParser } from "xmldom";
import { TestResult } from "./testResult";

function getAttributeValue(node, name: string): string {
const attribute = node.attributes.getNamedItem(name);
return (attribute === null) ? null : attribute.nodeValue;
}

export class TestResultsFile implements Disposable {
private static readonly ResultsFileName = "TestExplorerResults.trx";
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if we have multiple test project opened in different VS Code?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm now using a temporary random folder to store the results, so now works across multiple instances

private onNewResultsEmitter = new EventEmitter<TestResult[]>();
private resultsFile: string;
private watcher: fs.FSWatcher;

public constructor() {
const tempFolder = process.platform === "win32" ? process.env.TEMP : "/tmp";
this.resultsFile = path.join(tempFolder, TestResultsFile.ResultsFileName);

// The change event gets called multiple times, so use a one-second
// delay before we read anything to avoid doing too much work
const me = this;
let changeDelay: NodeJS.Timer;
this.watcher = fs.watch(tempFolder, (eventType, fileName) => {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only do this when 'showCodeLens' is true

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, done

if ((fileName === TestResultsFile.ResultsFileName) && (eventType === "change")) {
clearTimeout(changeDelay);
changeDelay = setTimeout(() => {
me.parseResults();
}, 1000);
}
});
}

public dispose(): void {
try {
this.watcher.close();
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also handle when when 'showCodeLens' is false

fs.unlinkSync(this.resultsFile);
} catch (error) {
}
}

public get fileName(): string {
return this.resultsFile;
}

public get onNewResults(): Event<TestResult[]> {
return this.onNewResultsEmitter.event;
}

private parseResults(): void {
const emitter = this.onNewResultsEmitter;
fs.readFile(this.resultsFile, (err, data) => {
if (!err) {
const results: TestResult[] = [];
const xdoc = new DOMParser().parseFromString(data.toString(), "application/xml");
const nodes = xdoc.documentElement.getElementsByTagName("UnitTestResult");

// TSLint wants to use for-of here, but nodes doesn't support it
for (let i = 0; i < nodes.length; i++) { // tslint:disable-line
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

@samcragg samcragg Nov 15, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did try that, however, at runtime it throws an exception TypeError: undefined is not a function as I don't believe HTMLCollection (returned by getElementsByTagName) has the iterator property so it can't be used in for-of. May be there's a better XML library I could use to parse the TRX file?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case, let's just keep this for loop.

results.push(new TestResult(
getAttributeValue(nodes[i], "testName"),
getAttributeValue(nodes[i], "outcome"),
));
}

emitter.fire(results);
}
});
}
}
25 changes: 25 additions & 0 deletions src/testStatusCodeLens.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
"use strict";
import { CodeLens, Range } from "vscode";

export class TestStatusCodeLens extends CodeLens {
public static parseOutcome(outcome: string): string {
if (outcome === "Passed") {
return "✔️";
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test on macOS, it is showing a black check mark (https://emojipedia.org/heavy-check-mark/) which is hard to see.
Two improvements:

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added settings now so the user can override what we display.

I've kept the heavy check mark for platforms apart from OS X, in which case it uses the white heavy check mark (I'm not a big fan of it, but I can't find any alternatives).

Since the user can specify the text, they could even switch to "passing", "FAILING" etc if they want so hopefully this is OK?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great to me!

} else if (outcome === "Failed") {
return "❌";
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, using unicode: https://emojipedia.org/cross-mark/

} else if (outcome === "NotExecuted") {
return "️️⚠️";
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, using unicode

} else {
return null;
}
}

public constructor(range: Range, status: string) {
super(range);

this.command = {
title: status,
command: null,
};
}
}
78 changes: 78 additions & 0 deletions src/testStatusCodeLensProvider.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
"use strict";
import { CancellationToken, CodeLens, CodeLensProvider, commands, Disposable, Event, EventEmitter, Range, SymbolInformation, SymbolKind, TextDocument, workspace } from "vscode";
import { TestResult } from "./testResult";
import { TestResultsFile } from "./testResultsFile";
import { TestStatusCodeLens } from "./testStatusCodeLens";

export class TestStatusCodeLensProvider implements CodeLensProvider {
private disposables: Disposable[] = [];
private enabled: boolean;
private onDidChangeCodeLensesEmitter = new EventEmitter<void>();

// Store everything in a map so we can remember old tests results for the
// scenario where a single test is ran. If the test no longer exists in
// code it will never be mapped to the symbol, so no harm (though there is
// a memory impact)
private testResults = new Map<string, TestResult>();

public constructor(testResultFile: TestResultsFile) {
this.checkEnabledOption();

this.disposables.push(
testResultFile.onNewResults(this.addTestResults, this));

this.disposables.push(
workspace.onDidChangeConfiguration(this.checkEnabledOption, this));
}

public dispose() {
while (this.disposables.length) {
this.disposables.pop().dispose();
}
}

public get onDidChangeCodeLenses(): Event<void> {
return this.onDidChangeCodeLensesEmitter.event;
}

public provideCodeLenses(document: TextDocument, token: CancellationToken): CodeLens[] | Thenable<CodeLens[]> {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Try with xUnit, NUnit and MSTest. Seems only xUnit is showing the test result icon. It would be great if you could also support NUnit and MSTest. 😃

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should now be fixed in the latest version, I've tried xUnit, MSTest and NUnit (the later two have the same format)

if (!this.enabled) {
return [];
}

const results = this.testResults;
return commands.executeCommand<SymbolInformation[]>("vscode.executeDocumentSymbolProvider", document.uri)
.then((symbols) => {
const mapped: CodeLens[] = [];
for (const symbol of symbols.filter((x) => x.kind === SymbolKind.Method)) {
const fullName = symbol.containerName + "." + symbol.name;
for (const result of results.values()) {
if (result.testName.endsWith(fullName)) {
Copy link
Owner

@formulahendry formulahendry Nov 15, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a corner case (when only namespace is different):
image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The symbol provider doesn't return anything for the namespace and the container of the class is empty, so would need to look at perhaps using omni-sharp directly? There's actually an issue with the test discovery with NUnit that only lists the method names, so if two tests exist with the same method name in different classes, NUnit only returns one of them.

Not sure how you want to approach this one? Accept it as a corner case until the extension uses omni-sharp to parse the files to discover the unit test methods?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this corner case could be accepted.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you have interest to look at omni-sharp or other approach, it is warmly welcome! You could send PR in the future.

const state = TestStatusCodeLens.parseOutcome(result.outcome);
if (state) {
mapped.push(new TestStatusCodeLens(symbol.location.range, state));
}
}
}
}

return mapped;
});
}

public resolveCodeLens(codeLens: CodeLens, token: CancellationToken): CodeLens {
return codeLens;
}

private addTestResults(results: TestResult[]) {
for (const result of results) {
this.testResults.set(result.testName, result);
}

this.onDidChangeCodeLensesEmitter.fire();
}

private checkEnabledOption(): void {
this.enabled = workspace.getConfiguration("csharp").get<boolean>("testsCodeLens.enabled", true);
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure it is better to add an extra boolean setting, in case user want to only show the run test | debug test. When user click run test, our test result icon would not updated. This may confuse users.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added an extra setting dotnet-test-explorer.showCodeLens

}
}