feat: deno test --docs support

[iykekings]

Closed #6124 in favour of this.

This PR adds --docs to deno test sub-command for running tests on JSDoc @example blocks.

To use:

have code blocks in @example sections with three back-ticks as in markdown code blocks
deno test --docs runs all non-test files
Imports are done relative to root path, eg.
src/linkedlist.ts

export class Linkedlist<T>  {
// skipped lines
 /**
 * @param list - LinkedList<T>
 * @example <caption>Linkedlist.compareWith</caption>
 * import { LinkedList } from './src/linkedlist.ts';
 * const testArr = [1, 2, 3, 4, 5, 6, 78, 9, 0, 65];
 * const firstList = new LinkedList<number>();
 * const secondList = new LinkedList<number>();
 * for (let data of testArr) {
 *   firstList.insertNode(data);
 *   secondList.insertNode(data);
 * }
 * const result = firstList.compareWith(secondList);
 * assert.assertEquals(result, true);
 * @returns boolean
 */
  compareWith(list: LinkedList<T>): boolean {
    let current1 = this.head;
    let current2 = list.head;
    while (current1 && current2) {
      if (current1.data !== current2.data) return false;
      if (current1.next && !current2.next && !current1.next && current2.next) {
        return false;
      }
      current1 = current1.next;
      current2 = current2.next;
    }
    return true;
  }
}

• All functions exported from https://deno.land/std/testing/asserts.ts can be used without importing them in an assert object. (assert.assert, assert.assertEquals, assert.assertNotEquals etc.)

• await keyword can be used without wrapping in async function. NB: This is not due to TLA, but await will be detected in the example and the generated test is wrapped in an async function.

• supports caption tag on @example block and used name of the test, by default test result is printed like this test src/linkedlist.ts - (line 160) ... ok (0ms), but with caption, like in the example above it will be test src/linkedlist.ts - Linkedlists.compareWith (line 160) ... ok (0ms)

• Supports multiple @example blocks in one JSDoc comment

To ignore test:

• wrap @example code block with back-ticks

• Tag the three opening back-ticks with ignore, and it will be picked up but not tested, will be recorded by Deno test as ignored.

• Tag the three opening back-ticks with text, and it will be seen as normal text

This is inspired by rustdoc --test

Example output

Resolves #4716

[bartlomieju]

Not really sure what's going on here, @iykekings please explain

[iykekings]

This is like opposite of is_supported for normal test files. It ensures doc tests are not run on test files and also that the file format is of normal expected type

[bartlomieju]

Those utils seems a little brittle 😬 we should probably leverage SWC for this purpose, but I guess it's alright for first pass

[iykekings]

Yeah I think so too, will have to parse each of the examples with swc

[bartlomieju]

This PR requires more work before it can be landed; mainly it needs to use SWC for parsing instead of regexes.

I'm removing this PR from 1.5.0 milestone - instead it will be landed in 1.5.x. This feature is hidden behind --unstable flag so we can land it anytime.

[wperron]

I'm just not sure about auto-importing asserts; it'd be nice to be able to simply copy paste the code block from the doc and run it as-is, as it stands one would need to add import * as assert from "https://deno.land/std/testing/asserts.ts" to the example block

[bartlomieju]

I'm just not sure about auto-importing asserts; it'd be nice to be able to simply copy paste the code block from the doc and run it as-is, as it stands one would need to add import * as assert from "https://deno.land/std/testing/asserts.ts" to the example block

@wperron I've already requested to remove this feature, I think it's not included in the PR anymore?

[bartlomieju]

Closing as stale because of numerous changes to test runner since this PR was open.

Thank you @iykekings this PR was a great base for @caspervonb, who's working on a new version with simpler API.

[iykekings]

Thanks a lot @bartlomieju
@caspervonb 🔥👍