Cannot exit Karma server gracefully

[filipesilva]

When the Karma server is run programmatically, there is no way to request it to stop and be informed of when that has happened:

const karma = require('karma');
const karmaServer = new karma.Server({ singleRun: false, configFile: __dirname + '/karma.conf.js' });
karmaServer.start();
// How to stop the server, and be informed when cleanup is finished?
// karma.stop(() => console.log('server exited gracefully'));

To reproduce, follow these steps:

git clone https://github.com/filipesilva/karma-exit-gracefully
cd karma-exit-gracefully
npm install
npm test

Having a way to gracefully stop Karma is useful when the Karma run is part of more complex orchestration, like in gulp or @angular/cli.

There is a way to stop the server by calling karma stop, but that will kill the process altogether:

karma/lib/middleware/stopper.js

Lines 11 to 13 in 6742ecf

	log.info('Stopping server')
	response.end('OK')
	process.kill(process.pid, 'SIGINT')

There's some cleanup logic as well, but it only runs on --single-run:

karma/lib/server.js

Lines 289 to 292 in 6742ecf

	this.on('run_complete', function (browsers, results) {
	this.log.debug('Run complete, exiting.')
	disconnectBrowsers(results.exitCode)
	})

Related to #2867, ampproject/amphtml#14814

Environment Details

• Karma version (output of karma --version): 3.0.0
• Relevant part of your karma.config.js file: included in repro, but nothing relevant in the config really.

[johnjbarton]

What is the purpose of stopping the server without exiting the process?

[filipesilva]

@johnjbarton running Karma as part of a larger pipeline. In a gulp task, you might want to do something after the server is stopped.

Inside Angular CLI, we orchestrate build tasks as well. We cannot test karma plugin behavior on rebuilds because we are unable to gracefully stop karma.

A specific usecase is that we want to test that our karma setup works on rebuilds, recovers from compilation errors, and does not trigger tests after a compilation with no changes. We set it up in https://github.com/angular/angular-cli/blob/master/packages/angular_devkit/build_angular/test/karma/rebuilds_spec_large.ts.

While we can test other toolchains this way, we cannot test karma because it would cause our test runner to exit.

[filipesilva]

@johnjbarton I added a tentative feature of how this could work in #3153.

A user could call use it this way:

const karma = require('karma');
const karmaServer = new karma.Server({ singleRun: false, configFile: __dirname + '/karma.conf.js' });

karmaServer.start()
  // do things
  .then(() => karmaServer.stop())
  .then(() => console.log('server exited gracefully'))

WDYT?