Not respecting Mocha version

[julian-sf]

Current behavior:

When attempting to override mocha version AND use an external reporter, cypress still internally loads mocha 2.5.3 which causes issues when attempting to use the reporters (I'll explain more below).

Desired behavior:

When specifying a mocha version, cypress uses that mocha version for ALL mocha-related activities, including Suite creation.

Steps to reproduce: (app code and test code)

package.json

{
  ...
  "devDependencies": {
    "cypress": "^3.3.2",
    "cypress-multi-reporters": "^1.1.22",
    "mocha": "^6.1.4"
  }
}

cypress.json

{
  "reporter": "cypress-multi-reporters",
  "reporterOptions": {
    "configFile": "reporter-config.json"
  }
}

reporter-config.json

{
  "reporterEnabled": "spec, xunit"
}

I first tried this with mocha-multi-reporters. This failed and is expected as per #3537. Fortunately, @YOU54F created a nice workaround in the package cypress-multi-reporters. Once I get past the issue in #3537; however, I run into a new issue when generating the report:

TypeError: test.isPending is not a function
    at XUnit.test (/Users/julian/dev/cypress-mocha-xunit/node_modules/mocha/lib/reporters/xunit.js:182:19)
    at /Users/julian/dev/cypress-mocha-xunit/node_modules/mocha/lib/reporters/xunit.js:106:12
    at Array.forEach (<anonymous>:null:null)
    at Runner.<anonymous> (/Users/julian/dev/cypress-mocha-xunit/node_modules/mocha/lib/reporters/xunit.js:105:11)
    at Object.onceWrapper (events.js:315:30)
    at emitOne (events.js:121:20)
    at Runner.emit (events.js:211:7)
    at Reporter.emit (/Users/julian/Library/Caches/Cypress/3.3.2/Cypress.app/Contents/Resources/app/packages/server/lib/reporter.js:239:55)
    at Object.server.startWebsockets.onMocha (/Users/julian/Library/Caches/Cypress/3.3.2/Cypress.app/Contents/Resources/app/packages/server/lib/project.js:300:22)
    at Socket.<anonymous> (/Users/julian/Library/Caches/Cypress/3.3.2/Cypress.app/Contents/Resources/app/packages/server/lib/socket.js:236:36)
    at emitTwo (events.js:126:13)
    at Socket.emit (events.js:214:7)
    at /Users/julian/Library/Caches/Cypress/3.3.2/Cypress.app/Contents/Resources/app/packages/socket/node_modules/socket.io/lib/socket.js:528:12
    at _combinedTickCallback (internal/process/next_tick.js:131:7)
    at process._tickCallback (internal/process/next_tick.js:180:9)

I've determined that this is specific to using the xunit built-in mocha reporter. When using,

reporter-config.json

{
  "reporterEnabled": "spec, junit"
}

everything works fine. What seems to happen is that cypress kicks off a test run and creates a Mocha.Suite. It then uses that suite to help collect the report and when it sends the Suite over to the mocha 6 built-in xunit reporter the above error occurs.

The old version of mocha uses a different Suite object and is missing the isPending method. What I can't seem to figure out is why cypress uses the internal mocha version and not the one installed in node_modules. I validated that the Suite is created with the internal mocha lib by placing the following line of code in the createSuite function @ packages/server/lib/reporter.js:56:

 createSuite = function(obj, parent) {
    var suite;
    console.log(require.resolve('mocha'));  // added this line
    suite = new Mocha.Suite(obj.title, {});
    if (parent) {
      suite.parent = parent;
    }
    if (obj.file) {
      console.log('has file:', obj.file);
    }
    suite.file = obj.file;
    return suite;
  };

Which results in (the bundled 2.4.5 version of mocha):

/Users/julian/Library/Caches/Cypress/3.3.2/Cypress.app/Contents/Resources/app/packages/server/node_modules/mocha/index.js

I checked the version like so:

➜  ~ cat /Users/julian/Library/Caches/Cypress/3.3.2/Cypress.app/Contents/Resources/app/packages/server/node_modules/mocha/package.json | grep version\":
  "version": "2.4.5"

Problem is, this is not the mocha version I specified... Is this expected behavior? Feels like it should respect the installed mocha version.

Versions

"cypress": "^3.3.2",
"cypress-multi-reporters": "^1.1.22",
"mocha": "^6.1.4"

Mac OS X 10.14.5

[julian-sf]

Repository with test code here: https://github.com/julian-sf/cypress-mocha-bug/tree/master

[julian-sf]

Note: The reason I think it works with junit and not xunit is most likely because junit is using mocha-junit-reporter under the hood, which somehow has the mocha version respected??? I don't know I'm grasping at straws here.

[cypress-app-bot]

This issue has not had any activity in 180 days. Cypress evolves quickly and the reported behavior should be tested on the latest version of Cypress to verify the behavior is still occurring. It will be closed in 14 days if no updates are provided.

[cypress-app-bot]

This issue has been closed due to inactivity.