Skip to content

Commit 4c8b855

Browse files
RunDevelopmentRunDevelopment
authored
Tests: Added --insert and --update parameters to language test (#2809)
1 parent ea82478 commit 4c8b855

File tree

4 files changed

+81
-46
lines changed

4 files changed

+81
-46
lines changed

‎test-suite.html

+38-13
Original file line numberDiff line numberDiff line change
@@ -85,22 +85,20 @@ <h2 id="writing-tests-creating-your-test-case-file">Creating your test case file
8585

8686

8787
<h2 id="writing-tests-writing-your-test">Writing your test</h2>
88-
<p>The structure of a test case file is as follows:</p>
89-
<pre><code>
90-
... language snippet...
91-
----
92-
... the simplified token stream you expect ...</code></pre>
9388

94-
<p>Your file is built up of two or three sections, separated by ten or more dashes <code>-</code>, starting at the begin of the line:</p>
89+
<p>A test case file is built up of two or three sections separated by ten or more dashes <code>-</code> starting at the begin of the line. The sections are the following:</p>
90+
9591
<ol>
9692
<li>Your language snippet. The code you want to tokenize using Prism. (<strong>required</strong>)</li>
9793
<li>
9894
The simplified token stream you expect. Needs to be valid JSON. (<em>optional</em>) <br>
99-
If there no token stream defined, the test case will fail unless the <code>--accept</code> flag is present when running the test command (e.g. <code>npm run test:languages -- --accept</code>). If the flag is present and there is no expected token stream, the runner will insert the actual token stream into the test case file, changing it.
95+
Instead of manually inserting the expected token stream yourself, prefer using the <code class="language-bash">npm run test:languages -- --insert</code> command. You can read more about this and related commands <a href="#writing-tests-insert-and-update">here</a>. <br>
96+
If there no token stream defined, the test case will fail unless the <code>--insert</code> or <code>--update</code> flag is present when running the test command.
10097
</li>
101-
<li>A comment explaining the test case. (<em>optional</em>)</li>
98+
<li>A brief comment explaining the test case. (<em>optional</em>)</li>
10299
</ol>
103-
<p>The easiest way would be to look at an existing test file:</p>
100+
101+
<p>Here is an example:</p>
104102
<pre><code>var a = 5;
105103

106104
----------------------------------------------------
@@ -117,21 +115,48 @@ <h2 id="writing-tests-writing-your-test">Writing your test</h2>
117115

118116
This is a comment explaining this test case.</code></pre>
119117

120-
<h2 id="writing-tests-the-easy-way">The easy way</h2>
118+
<h2 id="writing-tests-the-easy-way">The easy way to write tests</h2>
121119
<p>The easy way to create one or multiple new test case(s) is this:</p>
122120

123121
<ol>
124-
<li>Create a new file for a new test case in <code>tests/languages/${language}</code>.</li>
122+
<li>Create a new test case file <code class="language-none">tests/languages/{language}/{test-case}.test</code>.</li>
125123
<li>Insert the code you want to test (and nothing more).</li>
126124
<li>Repeat the first two steps for as many test cases as you want.</li>
127-
<li>Run <code>npm run test:languages -- --accept</code>.</li>
125+
<li>Run <code class="language-bash">npm run test:languages -- --insert</code>.</li>
126+
<li>Done.</li>
127+
</ol>
128+
129+
<p>Updating existing test case files is easy too!</p>
130+
131+
<ol>
132+
<li>Run <code class="language-bash">npm run test:languages -- --update</code>.</li>
128133
<li>Done.</li>
129134
</ol>
130135

131136
<p>This works by making the test runner insert the actual token stream of you test code as the expected token stream. <strong>Carefully check that the inserted token stream is actually what you expect or else the test is meaningless!</strong></p>
132137

133-
<p>Optionally, you can then also add comments to test cases.</p>
138+
<p>More details about the command can be found <a href="#writing-tests-insert-and-update">here</a>.</p>
139+
140+
<h2 id="writing-tests-insert-and-update">Insert and update expected token streams</h2>
141+
142+
<p>When creating and changing languages, their test files have to be updated to properly test the language. The rather tedious task of updating test files can be automated using the following commands:</p>
143+
144+
<ul>
145+
<li>
146+
<pre><code class="language-bash">npm run test:languages -- --insert</code></pre>
147+
148+
<p>This will insert the current actual token stream into all test files without an expected token stream. Test files that have an expected token stream are not affected.</p>
149+
150+
<p>This command is intended to be used when you want to create new test files while not updating existing ones.</p>
151+
</li>
152+
<li>
153+
<pre><code class="language-bash">npm run test:languages -- --update</code></pre>
154+
155+
<p>Updates (overwrites) the expected token stream of all failing test files and all test files that do not have an expected token stream. The language tests are guaranteed to pass after running this command.</p>
156+
</li>
157+
</ul>
134158

159+
<p><em>Keep in mind:</em> Both commands make it easy to create/update test files but this doesn't mean that the tests will be correct. <strong>Always carefully check the inserted/updated token streams!</strong></p>
135160

136161
<h2 id="writing-tests-explaining-the-simplified-token-stream">Explaining the simplified token stream</h2>
137162

‎tests/helper/test-case.js

+32-23
Original file line numberDiff line numberDiff line change
@@ -53,9 +53,9 @@ module.exports = {
5353
*
5454
* @param {string} languageIdentifier
5555
* @param {string} filePath
56-
* @param {boolean} acceptEmpty
56+
* @param {"none" | "insert" | "update"} updateMode
5757
*/
58-
runTestCase(languageIdentifier, filePath, acceptEmpty) {
58+
runTestCase(languageIdentifier, filePath, updateMode) {
5959
const testCase = this.parseTestCaseFile(filePath);
6060
const usedLanguages = this.parseLanguageNames(languageIdentifier);
6161

@@ -64,30 +64,31 @@ module.exports = {
6464
// the first language is the main language to highlight
6565
const tokenStream = this.tokenize(Prism, testCase.code, usedLanguages.mainLanguage);
6666

67-
if (testCase.expectedTokenStream === null) {
68-
// the test case doesn't have an expected value
69-
if (!acceptEmpty) {
70-
throw new Error('This test case doesn\'t have an expected toke n stream.'
71-
+ ' Either add the JSON of a token stream or run \`npm run test:languages -- --accept\`'
72-
+ ' to automatically add the current token stream.');
73-
}
74-
67+
function updateFile() {
7568
// change the file
7669
const lineEnd = (/\r\n/.test(testCase.code) || !/\n/.test(testCase.code)) ? '\r\n' : '\n';
77-
const separator = "\n\n----------------------------------------------------\n\n";
78-
const pretty = TokenStreamTransformer.prettyprint(tokenStream)
79-
.replace(/^( +)/gm, m => {
80-
return "\t".repeat(m.length / 4);
81-
});
70+
const separator = '\n\n----------------------------------------------------\n\n';
71+
const pretty = TokenStreamTransformer.prettyprint(tokenStream, '\t');
8272

8373
let content = testCase.code + separator + pretty;
84-
if (testCase.comment) {
85-
content += separator + testCase.comment;
74+
if (testCase.comment.trim()) {
75+
content += separator + testCase.comment.trim();
8676
}
87-
//content += '\n'
77+
content += '\n'
8878
content = content.replace(/\r?\n/g, lineEnd);
8979

90-
fs.writeFileSync(filePath, content, "utf-8");
80+
fs.writeFileSync(filePath, content, 'utf-8');
81+
}
82+
83+
if (testCase.expectedTokenStream === null) {
84+
// the test case doesn't have an expected value
85+
if (updateMode === 'none') {
86+
throw new Error('This test case doesn\'t have an expected token stream.'
87+
+ ' Either add the JSON of a token stream or run \`npm run test:languages -- --insert\`'
88+
+ ' to automatically add the current token stream.');
89+
}
90+
91+
updateFile();
9192
} else {
9293
// there is an expected value
9394
const simplifiedTokenStream = TokenStreamTransformer.simplify(tokenStream);
@@ -100,6 +101,11 @@ module.exports = {
100101
return;
101102
}
102103

104+
if (updateMode === 'update') {
105+
updateFile();
106+
return;
107+
}
108+
103109
// The index of the first difference between the expected token stream and the actual token stream.
104110
// The index is in the raw expected token stream JSON of the test case.
105111
const diffIndex = translateIndexIgnoreSpaces(testCase.expectedJson, expected, firstDiff(expected, actual));
@@ -108,11 +114,14 @@ module.exports = {
108114
const lineNumber = testCase.expectedLineOffset + expectedJsonLines.length;
109115

110116
const tokenStreamStr = TokenStreamTransformer.prettyprint(tokenStream);
111-
const message = "\n\nActual Token Stream:" +
112-
"\n-----------------------------------------\n" +
117+
const message = `\nThe expected token stream differs from the actual token stream.` +
118+
` Either change the ${usedLanguages.mainLanguage} language or update the expected token stream.` +
119+
` Run \`npm run test:languages -- --update\` to update all missing or incorrect expected token streams.` +
120+
`\n\n\nActual Token Stream:` +
121+
`\n-----------------------------------------\n` +
113122
tokenStreamStr +
114-
"\n-----------------------------------------\n" +
115-
"File: " + filePath + ":" + lineNumber + ":" + columnNumber + "\n\n";
123+
`\n-----------------------------------------\n` +
124+
`File: ${filePath}:${lineNumber}:${columnNumber}\n\n`;
116125

117126
assert.deepEqual(simplifiedTokenStream, testCase.expectedTokenStream, testCase.comment + message);
118127
}

‎tests/helper/token-stream-transformer.js

+8-8
Original file line numberDiff line numberDiff line change
@@ -50,10 +50,11 @@ module.exports = {
5050

5151
/**
5252
* @param {TokenStream} tokenStream
53+
* @param {string} [indentation]
5354
* @returns {string}
5455
*/
55-
prettyprint(tokenStream) {
56-
return printPrettyTokenStream(toPrettyTokenStream(tokenStream));
56+
prettyprint(tokenStream, indentation) {
57+
return printPrettyTokenStream(toPrettyTokenStream(tokenStream), undefined, indentation);
5758
}
5859
};
5960

@@ -219,13 +220,12 @@ function prettyFormat(prettyStream) {
219220
/**
220221
* @param {PrettyTokenStream} prettyStream
221222
* @param {number} [indentationLevel]
223+
* @param {string} [indentationChar]
222224
* @returns {string}
223225
*/
224-
function printPrettyTokenStream(prettyStream, indentationLevel = 1) {
225-
const indentChar = ' ';
226-
226+
function printPrettyTokenStream(prettyStream, indentationLevel = 1, indentationChar = ' ') {
227227
// can't use tabs because the console will convert one tab to four spaces
228-
const indentation = new Array(indentationLevel + 1).join(indentChar);
228+
const indentation = new Array(indentationLevel + 1).join(indentationChar);
229229

230230
let out = '';
231231
out += '[\n';
@@ -265,7 +265,7 @@ function printPrettyTokenStream(prettyStream, indentationLevel = 1) {
265265
out += JSON.stringify(content);
266266
} else {
267267
// token stream
268-
out += printPrettyTokenStream(content, indentationLevel + 1);
268+
out += printPrettyTokenStream(content, indentationLevel + 1, indentationChar);
269269
}
270270

271271
out += ']';
@@ -275,7 +275,7 @@ function printPrettyTokenStream(prettyStream, indentationLevel = 1) {
275275
out += lineEnd;
276276
}
277277
})
278-
out += indentation.substr(indentChar.length) + ']'
278+
out += indentation.substr(indentationChar.length) + ']'
279279
return out;
280280
}
281281

‎tests/run.js

+3-2
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,8 @@ const testSuite =
1212
// load complete test suite
1313
: TestDiscovery.loadAllTests(__dirname + "/languages");
1414

15-
const accept = !!argv.accept;
15+
const insert = !!argv.accept || !!argv.insert;
16+
const update = !!argv.update;
1617

1718
// define tests for all tests in all languages in the test suite
1819
for (const language in testSuite) {
@@ -29,7 +30,7 @@ for (const language in testSuite) {
2930

3031
it("– should pass test case '" + fileName + "'", function () {
3132
if (path.extname(filePath) === '.test') {
32-
TestCase.runTestCase(language, filePath, accept);
33+
TestCase.runTestCase(language, filePath, update ? "update" : insert ? "insert" : "none");
3334
} else {
3435
TestCase.runTestsWithHooks(language, require(filePath));
3536
}

0 commit comments

Comments
 (0)