Failing tests no longer produce stack traces with clickable file links

[RonanQuigley]

Prerequisites

• Checked that your issue hasn't already been filed by cross-referencing issues with the faq label
• Checked next-gen ES issues and syntax problems by using the same environment and/or transpiler configuration without Mocha to ensure it isn't just a feature that actually isn't supported in the environment in question or a bug in your code.
• 'Smoke tested' the code to be tested by running it outside the real test suite to get a better sense of whether the problem is in the code under test, your usage of Mocha, or Mocha itself
• Ensured that there is no discrepancy between the locally and globally installed versions of Mocha. You can find them with: node node_modules/.bin/mocha --version(Local) and mocha --version(Global). We recommend avoiding the use of globally installed Mocha.

Description

Consider the following test file. It is written in common js and uses no transpilers. With an earlier version of mocha that I used (2.2.5), failing tests produced a stack trace that had clickable links (at least for vscode) to the offending file when a unit test failed. After trying mocha 5.1.1 with this same example test file it no longer works:

const chai = require('chai');

const expect = chai.expect;

function foo() {
    return 'foo';
}

describe('failing test', () => {
    it('should return bar', () => {
        expect(foo()).to.equal('bar');
    });
});
 

Steps to Reproduce

Expected behavior: Mocha should be able to produce stack traces with clickable links inside a shell/terminal (using vscode with bash) to the line and file of a failing test

Actual behavior: The stack trace file ref is no longer clickable:

Reproduces how often: Always

Versions

• The output of mocha --version and node node_modules/.bin/mocha --version: 5.1.1
• The output of node --version: 10.1.0
• The version and architecture of your operating system: windows 10, 64 bit
• Your shell (bash, zsh, PowerShell, cmd, etc.): bash
• Your browser and version (if running browser tests): N/A
• Any other third party Mocha related modules (with versions): N/A
• The code transpiler being used: babel

Additional Information

This issue has also been replicated in my own current project that uses babel. In babel's case the link looks different; it seems to log it twice?

To get around the problem, I've resorted to using mocha-webpack, as that does provide clickable file links.

[plroebuck]

Since you jumped several major versions at once, did you try updating one major version at a time, then one minor version, then one patch version? If you can tell us exactly which release it actually stopped working, it would be more helpful trying to determine why it did. Would appear from your screenshots that at some point, only the basename was printed rather than the full path.

BTW, nice job on the issue report! Your first two screenshots demonstrate your problem very well.

[boneskull]

There was a recent change to the stacktrace-fiddling code which is a likely culprit.

[Munter]

This looks like the relevant piece of code: https://github.com/mochajs/mocha/blob/master/lib/utils.js#L643-L646

[RonanQuigley]

I fiddled around with installing versions and on my end it started occurring from version 3.1.0. 3.0.2 Is fine, though somewhere between that version and 2.25 the stack traces are also became much more verbose in their output:

[plroebuck]

@boneskull, see PR #2503 (from Mocha-3.1.0) [stacktracelimit munging]

[Munter]

To be honest I would prefer if we could keep the path part of the stack trace small to simplify the human readability. I wonder if we can come up with a format that matches some of the regexes of different editors to add links to track traces. Here's the change in vs code that adds the ability to pick up col and line numbers: microsoft/vscode#24832

Or if the relevant scope of editors is not too large we could try to convince them of this use case :)

[plroebuck]

think this should be changed to 'confirmed bug'

[boneskull]

It's confirmed bug if someone else has confirmed it, so please let me know if that's the case.

[RonanQuigley]

Have either one of you attempted a replication? It would move this confirmation along. I can provide a github repo with the issue if it helps.

[plroebuck]

I use vi on macOS. If it needs Windows to reproduce, I cannot help much past research.

[RonanQuigley]

Yeah I don't use mac so unsure if it's OS specifc. Though it would be worth seeing if this does also happen on macOS no? Could also just be vscode that does it.

[outsideris]

It seems a design decision rather than a bug.

In my OSX, stack trace path is relative path.

  1) failing test
       should return bar:

      AssertionError [ERR_ASSERTION]: 'foo' == 'bar'
      + expected - actual

      -foo
      +bar

      at Context.it (test.js:9:16)
      at callFn (lib/runnable.js:372:21)
      at Test.Runnable.run (lib/runnable.js:364:7)
      at Runner.runTest (lib/runner.js:455:10)
      at /Users/outsider/projects/github/mocha/lib/runner.js:573:12
      at next (lib/runner.js:369:14)
      at /Users/outsider/projects/github/mocha/lib/runner.js:379:7
      at next (lib/runner.js:303:14)
      at Immediate.<anonymous> (lib/runner.js:347:5

You can print full stack trace with --full-trace options.

  1) failing test
       should return bar:

      AssertionError [ERR_ASSERTION]: 'foo' == 'bar'
      + expected - actual

      -foo
      +bar

      at Context.it (/Users/outsider/projects/github/mocha/test.js:9:16)
      at callFn (/Users/outsider/projects/github/mocha/lib/runnable.js:372:21)
      at Test.Runnable.run (/Users/outsider/projects/github/mocha/lib/runnable.js:364:7)
      at Runner.runTest (/Users/outsider/projects/github/mocha/lib/runner.js:455:10)
      at /Users/outsider/projects/github/mocha/lib/runner.js:573:12
      at next (/Users/outsider/projects/github/mocha/lib/runner.js:369:14)
      at /Users/outsider/projects/github/mocha/lib/runner.js:379:7
      at next (/Users/outsider/projects/github/mocha/lib/runner.js:303:14)
      at Immediate.<anonymous> (/Users/outsider/projects/github/mocha/lib/runner.js:347:5)
      at runCallback (timers.js:789:20)
      at tryOnImmediate (timers.js:751:5)
      at processImmediate [as _immediateCallback] (timers.js:722:5)

Can this solve your problem?

[RonanQuigley]

Yes that fixes the issue with clickable file links, but at the expense of having a full trace. I tried using mocha-clean to compensate but that just puts you back to square one.

If it's a design decision this seems odd to me as it just leaves mocha with a usability problem. Is there no way to just have the best of both worlds; a clean minimal stack trace yet clickable file references at the same time? Unsure if your editor of choice allows for this, but are your osx file links clickable at all?

[plroebuck]

...but are your osx file links clickable at all?

There are no links in Mocha's stacktrace output. vscode is creating those links for you.

[RonanQuigley]

Oops I forgot about this, and the email notification for this thread from that last comment popped into my head the fact that this fixed itself back at the end of June with vscode's update. Feel free to close this issue now.