diff --git a/packages/stryker/.vscode/launch.json b/packages/stryker/.vscode/launch.json index 5d87b722b7..8a8337fe5f 100644 --- a/packages/stryker/.vscode/launch.json +++ b/packages/stryker/.vscode/launch.json @@ -77,7 +77,7 @@ "run", "stryker.conf.js" ], - "cwd": "${workspaceRoot}", + "cwd": "${workspaceRoot}/../../integrationTest/test/typescript-transpiling", "runtimeExecutable": null, "runtimeArgs": [ "--nolazy" diff --git a/packages/stryker/README.md b/packages/stryker/README.md index 23eb6cb75c..b505c5d5e1 100644 --- a/packages/stryker/README.md +++ b/packages/stryker/README.md @@ -47,11 +47,10 @@ The following is an example `stryker.conf.js` file: ```javascript module.exports = function(config){ config.set({ - files: ['test/helpers/**/*.js', - 'test/unit/**/*.js', - { pattern: 'src/**/*.js', included: false, mutated: true } - { pattern: 'src/templates/*.html', included: false, mutated: false } - '!src/fileToIgnore.js'], + mutate: [ + 'src/**/*.js' + '!src/index.js' + ], testFramework: 'mocha', testRunner: 'mocha', reporter: ['progress', 'clear-text', 'dots', 'html', 'event-recorder'], @@ -61,61 +60,43 @@ module.exports = function(config){ } ``` -As you can see, the config file is *not* a simple JSON file. It should be a common js (a.k.a. node) module. You might recognize this way of working from the karma test runner. +As you can see, the config file is *not* a simple JSON file. It should be a node module. You might recognize this way of working from the karma test runner. -Make sure you *at least* specify the `files` and the `testRunner` options when mixing the config file and/or command line options. +Make sure you *at least* specify the `testRunner` options when mixing the config file and/or command line options. ## Command-line interface Stryker can also be installed, configured and run using the [Stryker-CLI](https://github.com/stryker-mutator/stryker-cli). If you plan on using Stryker in more projects, the Stryker-CLI is the easiest way to install, configure and run Stryker for your project. You can install the Stryker-CLI using: -``` +```bash $ npm install -g stryker-cli ``` The Stryker-CLI works by passing received commands to your local Stryker installation. If you don't have Stryker installed yet, the Stryker-CLI will help you with your Stryker installation. This method allows us to provide additional commands with updates of Stryker itself. -## Supported mutators +## Supported mutators + See our website for the [list of currently supported mutators](https://stryker-mutator.io/mutators.html). ## Configuration + All configuration options can either be set via the command line or via the `stryker.conf.js` config file. `files` and `mutate` both support globbing expressions using [node glob](https://github.com/isaacs/node-glob). This is the same globbing format you might know from [Grunt](https://github.com/gruntjs/grunt) or [Karma](https://github.com/karma-runner/karma). -You can *ignore* files by adding an exclamation mark (`!`) at the start of an expression. - -#### Files required to run your tests -**Command line:** `[--files|-f] node_modules/a-lib/**/*.js,src/**/*.js,a.js,test/**/*.js` -**Config file:** `files: ['{ pattern: 'src/**/*.js', mutated: true }, '!src/**/index.js', 'test/**/*.js']` -**Default value:** *none* -**Mandatory**: yes -**Description:** -With `files` you specify all files needed to run your tests. If the test runner you use already provides the test framework (Jasmine, Mocha, etc.), -you should *not* include those files here as well. -The files will be loaded in the order in which they are specified. Files that you want to ignore should be mentioned last. - -When using the command line, the list can only contain a comma separated list of globbing expressions. -When using the config file you can provide an array with `string`s or `InputFileDescriptor` objects, like so: - -* `string`: The globbing expression used for selecting the files needed to run the tests. -* `InputFileDescriptor` object: `{ pattern: 'pattern', included: true, mutated: false }`: - * The `pattern` property is mandatory and contains the globbing expression used for selecting the files. Using `!` to ignore files is *not* supported here. - * The `included` property is optional and determines whether or not this file should be loaded initially by the test-runner (default: true). With `included: false` the files will be copied to the sandbox during testing, but they wont be explicitly loaded by the test runner. Two usecases for `included: false` are for HTML files and for source files when your tests `require()` them. - * The `mutated` property is optional and determines whether or not this file should be targeted for mutations (default: false) -*Note*: To include a file/folder which start with an exclamation mark (`!`), use the `InputFileDescriptor` syntax. +You can *ignore* files by adding an exclamation mark (`!`) at the start of an expression. -#### Source code files to mutate +#### Files to mutate **Command line:** `[--mutate|-m] src/**/*.js,a.js` **Config file:** `mutate: ['src/**/*.js', 'a.js']` **Default value:** *none* -**Mandatory**: no +**Mandatory**: No **Description:** -With `mutate` you configure the subset of files to use for mutation testing. Generally speaking, these should be your own source files. -This is optional, as you can also use the `mutated` property with the `files` parameter or not mutate any files at all to perform a dry-run (test-run). -We expect a comma separated list of globbing expressions, which will be used to select the files to be mutated. +With `mutate` you configure the subset of files to use for mutation testing. +Generally speaking, these should be your own source files. +This is optional, as you can choose to not mutate any files at all and perform a dry-run (running only your tests without mutating). #### Test runner **Command line:** `--testRunner karma` @@ -131,10 +112,13 @@ See the [list of plugins](https://stryker-mutator.io/plugins.html) for an up-to- **Command line:** `--testFramework jasmine` **Config file:** `testFramework: 'jasmine'` **Default value:** *none* -**Mandatory**: yes +**Mandatory**: No **Description:** -With `testFramework` you configure which test framework your tests are using. This value is directly consumed by the test runner and therefore -depends what framework that specific test runner supports. By default, this value is also used for `testFramework`. +Configure which test framework you are using. +This option is not mandatory, as Stryker is test framework agnostic (it doesn't care what framework you use), +However, it is required when `coverageAnalysis` is set to `'perTest'`, because Stryker needs to hook into the test framework in order to measure code coverage results per test and filter tests to run. + +Make sure the a plugin is installed for your chosen test framework. E.g. install `stryker-mocha-framework` to use `'mocha'` as a test framework. #### Type of coverage analysis **Full notation:** `--coverageAnalysis perTest` @@ -142,7 +126,8 @@ depends what framework that specific test runner supports. By default, this valu **Default value:** `perTest` **Mandatory**: no **Description:** -With `coverageAnalysis` you specify which coverage analysis strategy you want to use. +With `coverageAnalysis` you specify which coverage analysis strategy you want to use. + Stryker can analyse code coverage results. This can potentially speed up mutation testing a lot, as only the tests covering a particular mutation are tested for each mutant. This does *not* influence the resulting mutation testing score. It only improves performance, so we enable it by default. @@ -212,6 +197,24 @@ The `dashboard` reporter is a special kind of reporter. It sends a report to htt All `TRAVIS` environment variables are set by Travis for each build. However, you will need to pass the `STRYKER\_DASHBOARD\_API\_KEY` environment variable yourself. You can create one for your repository by logging in on [the stryker dashboard](https://dashboard.stryker-mutator.io). We strongly recommend you use [encrypted environment variables](https://docs.travis-ci.com/user/environment-variables/#Encrypting-environment-variables). +#### Files in the sandbox +**Command line:** `[--files|-f] src/**/*.js,a.js,test/**/*.js` +**Config file:** `files: ['src/**/*.js', '!src/**/index.js', 'test/**/*.js']` +**Default value:** result of `git ls-files --others --exclude-standard --cached` +**Mandatory**: No +**Description:** +With `files` you can choose which files should be included in your test runner sandbox. +This is normally not needed as it defaults to all files not ignored by git. +Try it out yourself with this command: `git ls-files --others --exclude-standard --cached`. + +If you do need to override `files` (for example: when your project does not live in a git repository), +you can override the files here. + +When using the command line, the list can only contain a comma separated list of globbing expressions. +When using the config file you can provide an array with `string`s + +You can *ignore* files by adding an exclamation mark (`!`) at the start of an expression. + #### Plugins **Command line:** `--plugins stryker-html-reporter,stryker-karma-runner` **Config file:** `plugins: ['stryker-html-reporter', 'stryker-karma-runner']` @@ -287,8 +290,7 @@ Specify the thresholds for mutation score. * `mutation score < low`: Danger! Reporters should color this in red. You're in danger! * `mutation score < break`: Error! Stryker will exit with exit code 1, indicating a build failure. No consequence for reporters, though. -It is not allowed to only supply one value. However, `high` and `low` values can be the same, making sure colors are either red or green. -Set `break` to `null` (default) to never let the process crash. +It is not allowed to only supply one value of the values (it's all or nothing). However, `high` and `low` values can be the same, making sure colors are either red or green. Set `break` to `null` (default) to never let your build fail. #### Log level **Command line:** `--logLevel info` diff --git a/packages/stryker/package.json b/packages/stryker/package.json index d3618a1776..f0e6442328 100644 --- a/packages/stryker/package.json +++ b/packages/stryker/package.json @@ -68,8 +68,8 @@ "progress": "^2.0.0", "rimraf": "^2.6.1", "rxjs": "^5.4.3", - "serialize-javascript": "^1.3.0", "source-map": "^0.6.1", + "surrial": "^0.1.3", "tslib": "^1.5.0", "typed-rest-client": "^1.0.7" }, diff --git a/packages/stryker/src/InputFileResolver.ts b/packages/stryker/src/InputFileResolver.ts deleted file mode 100644 index 845096be06..0000000000 --- a/packages/stryker/src/InputFileResolver.ts +++ /dev/null @@ -1,225 +0,0 @@ -import * as os from 'os'; -import * as path from 'path'; -import * as fs from 'mz/fs'; -import * as _ from 'lodash'; -import { getLogger } from 'log4js'; -import { File, InputFileDescriptor, FileDescriptor, FileKind, TextFile, BinaryFile } from 'stryker-api/core'; -import { glob, isOnlineFile, determineFileKind } from './utils/fileUtils'; -import StrictReporter from './reporters/StrictReporter'; - -const DEFAULT_INPUT_FILE_PROPERTIES = { mutated: false, included: true, transpiled: true }; - -function testFileToReportFile(textFile: TextFile) { - return { - path: textFile.name, - content: textFile.content - }; -} - -export default class InputFileResolver { - - private readonly log = getLogger(InputFileResolver.name); - private inputFileResolver: PatternResolver; - private mutateResolver: PatternResolver; - - constructor(mutate: string[], allFileExpressions: Array, private reporter: StrictReporter) { - this.validateFileDescriptor(allFileExpressions); - this.validateMutationArray(mutate); - this.mutateResolver = PatternResolver.parse(mutate || []); - this.inputFileResolver = PatternResolver.parse(allFileExpressions); - } - - public async resolve(): Promise { - const [inputFileDescriptors, mutateFiles] = await Promise.all([this.inputFileResolver.resolve(), this.mutateResolver.resolve()]); - const files: File[] = await this.readFiles(inputFileDescriptors); - this.markAdditionalFilesToMutate(files, mutateFiles.map(m => m.name)); - this.logFilesToMutate(files); - this.reportAllSourceFilesRead(files); - return files; - } - - private validateFileDescriptor(maybeInputFileDescriptors: Array) { - maybeInputFileDescriptors.forEach(maybeInputFileDescriptor => { - if (_.isObject(maybeInputFileDescriptor)) { - if (Object.keys(maybeInputFileDescriptor).indexOf('pattern') === -1) { - throw Error(`File descriptor ${JSON.stringify(maybeInputFileDescriptor)} is missing mandatory property 'pattern'.`); - } else { - maybeInputFileDescriptor = maybeInputFileDescriptor as InputFileDescriptor; - if (isOnlineFile(maybeInputFileDescriptor.pattern) && maybeInputFileDescriptor.mutated) { - throw new Error(`Cannot mutate web url "${maybeInputFileDescriptor.pattern}".`); - } - } - } - }); - } - - private validateMutationArray(mutationArray: Array) { - if (mutationArray) { - mutationArray.forEach(mutation => { - if (isOnlineFile(mutation)) { - throw new Error(`Cannot mutate web url "${mutation}".`); - } - }); - } - } - - private markAdditionalFilesToMutate(allInputFiles: File[], additionalMutateFiles: string[]) { - const errors: string[] = []; - additionalMutateFiles.forEach(mutateFile => { - if (!allInputFiles.filter(inputFile => inputFile.name === mutateFile).length) { - errors.push(`Could not find mutate file "${mutateFile}" in list of files.`); - } - }); - if (errors.length > 0) { - throw new Error(errors.join(' ')); - } - allInputFiles.forEach(file => file.mutated = additionalMutateFiles.some(mutateFile => mutateFile === file.name) || file.mutated); - } - - private logFilesToMutate(allInputFiles: File[]) { - let mutateFiles = allInputFiles.filter(file => file.mutated); - if (mutateFiles.length) { - this.log.info(`Found ${mutateFiles.length} of ${allInputFiles.length} file(s) to be mutated.`); - } else { - this.log.warn(`No files marked to be mutated, stryker will perform a dry-run without actually mutating anything.`); - } - if (this.log.isDebugEnabled) { - this.log.debug('All input files in order:%s', allInputFiles.map(file => `${os.EOL}\t${file.name} (included: ${file.included}, mutated: ${file.mutated})`)); - } - } - - private reportAllSourceFilesRead(allFiles: File[]) { - this.reporter.onAllSourceFilesRead(this.filterTextFiles(allFiles).map(testFileToReportFile)); - } - - private reportSourceFilesRead(textFile: TextFile) { - this.reporter.onSourceFileRead(testFileToReportFile(textFile)); - } - - private filterTextFiles(files: File[]): TextFile[] { - return files.filter(file => file.kind === FileKind.Text) as TextFile[]; - } - - private readFiles(inputFileDescriptors: FileDescriptor[]): Promise { - return Promise.all(inputFileDescriptors.map(file => this.readInputFile(file))); - } - - private readInputFile(descriptor: FileDescriptor): Promise { - switch (descriptor.kind) { - case FileKind.Web: - const web: { kind: FileKind.Web } = { kind: FileKind.Web }; - return Promise.resolve(Object.assign({}, descriptor, web)); - case FileKind.Text: - return this.readLocalFile(descriptor, descriptor.kind).then(textFile => { - this.reportSourceFilesRead(textFile as TextFile); - return textFile; - }); - default: - return this.readLocalFile(descriptor, descriptor.kind); - } - } - - private readLocalFile(descriptor: FileDescriptor, kind: FileKind.Text | FileKind.Binary): Promise { - return this.readInputFileContent(descriptor.name, kind).then(content => ({ - name: descriptor.name, - kind: descriptor.kind, - content, - included: descriptor.included, - mutated: descriptor.mutated, - transpiled: descriptor.transpiled - }) as TextFile | BinaryFile); - } - - private readInputFileContent(fileName: string, kind: FileKind.Binary | FileKind.Text): Promise { - if (kind === FileKind.Binary) { - return fs.readFile(fileName); - } else { - return fs.readFile(fileName, 'utf8'); - } - } -} - -class PatternResolver { - - private readonly log = getLogger(InputFileResolver.name); - private ignore = false; - private descriptor: InputFileDescriptor; - - constructor(descriptor: InputFileDescriptor | string, private previous?: PatternResolver) { - if (typeof descriptor === 'string') { // mutator array is a string array - this.descriptor = Object.assign({ pattern: descriptor }, DEFAULT_INPUT_FILE_PROPERTIES); - this.ignore = descriptor.indexOf('!') === 0; - if (this.ignore) { - this.descriptor.pattern = descriptor.substring(1); - } - } else { - this.descriptor = descriptor; - } - } - - async resolve(): Promise { - // When the first expression starts with an '!', we skip that one - if (this.ignore && !this.previous) { - return Promise.resolve([]); - } else { - // Start the globbing task for the current descriptor - const globbingTask = this.resolveGlobbingExpression(this.descriptor.pattern) - .then(filePaths => filePaths.map(filePath => this.createInputFile(filePath))); - - // If there is a previous globbing expression, resolve that one as well - if (this.previous) { - const results = await Promise.all([this.previous.resolve(), globbingTask]); - const previousFiles = results[0]; - const currentFiles = results[1]; - // If this expression started with a '!', exclude current files - if (this.ignore) { - return previousFiles.filter(previousFile => currentFiles.every(currentFile => previousFile.name !== currentFile.name)); - } else { - // Only add files which were not already added - return previousFiles.concat(currentFiles.filter(currentFile => !previousFiles.some(file => file.name === currentFile.name))); - } - } else { - return globbingTask; - } - } - } - - static empty(): PatternResolver { - const emptyResolver = new PatternResolver(''); - emptyResolver.ignore = true; - return emptyResolver; - } - - static parse(inputFileExpressions: Array): PatternResolver { - const expressions = inputFileExpressions.map(i => i); // work on a copy as we're changing the array state - let current = PatternResolver.empty(); - let expression = expressions.shift(); - while (expression) { - current = new PatternResolver(expression, current); - expression = expressions.shift(); - } - return current; - } - - private async resolveGlobbingExpression(pattern: string): Promise { - if (isOnlineFile(pattern)) { - return Promise.resolve([pattern]); - } else { - let files = await glob(pattern); - if (files.length === 0) { - this.reportEmptyGlobbingExpression(pattern); - } - return files.map((f) => path.resolve(f)); - } - } - - private reportEmptyGlobbingExpression(expression: string) { - this.log.warn(`Globbing expression "${expression}" did not result in any files.`); - } - - private createInputFile(name: string): FileDescriptor { - const inputFile = _.assign({ name, kind: determineFileKind(name) }, DEFAULT_INPUT_FILE_PROPERTIES, this.descriptor); - delete (inputFile)['pattern']; - return inputFile; - } -} \ No newline at end of file diff --git a/packages/stryker/src/MutantTestMatcher.ts b/packages/stryker/src/MutantTestMatcher.ts index 1c9936df73..de0fa9778e 100644 --- a/packages/stryker/src/MutantTestMatcher.ts +++ b/packages/stryker/src/MutantTestMatcher.ts @@ -1,7 +1,7 @@ import * as _ from 'lodash'; import { getLogger } from 'log4js'; import { RunResult, CoverageCollection, StatementMap, CoveragePerTestResult, CoverageResult } from 'stryker-api/test_runner'; -import { StrykerOptions, File, TextFile } from 'stryker-api/core'; +import { StrykerOptions, File } from 'stryker-api/core'; import { MatchedMutant } from 'stryker-api/report'; import { Mutant } from 'stryker-api/mutant'; import TestableMutant, { TestSelectionResult } from './TestableMutant'; @@ -31,8 +31,8 @@ export default class MutantTestMatcher { private readonly log = getLogger(MutantTestMatcher.name); constructor( - private mutants: Mutant[], - private files: File[], + private mutants: ReadonlyArray, + private filesToMutate: ReadonlyArray, private initialRunResult: RunResult, private sourceMapper: SourceMapper, private coveragePerFile: CoverageMapsByFile, @@ -56,7 +56,7 @@ export default class MutantTestMatcher { testableMutants.forEach(mutant => mutant.selectAllTests(this.initialRunResult, TestSelectionResult.Success)); } else if (!this.initialRunResult.coverage) { this.log.warn('No coverage result found, even though coverageAnalysis is "%s". Assuming that all tests cover each mutant. This might have a big impact on the performance.', this.options.coverageAnalysis); - testableMutants.forEach(mutant => mutant.selectAllTests(this.initialRunResult, TestSelectionResult.FailedButAlreadyReporter)); + testableMutants.forEach(mutant => mutant.selectAllTests(this.initialRunResult, TestSelectionResult.FailedButAlreadyReported)); } else { testableMutants.forEach(testableMutant => this.enrichWithCoveredTests(testableMutant)); } @@ -118,7 +118,7 @@ export default class MutantTestMatcher { } private createTestableMutants(): TestableMutant[] { - const sourceFiles = this.files.filter(file => file.mutated).map(file => new SourceFile(file as TextFile)); + const sourceFiles = this.filesToMutate.map(file => new SourceFile(file)); return filterEmpty(this.mutants.map((mutant, index) => { const sourceFile = sourceFiles.find(file => file.name === mutant.fileName); if (sourceFile) { diff --git a/packages/stryker/src/MutatorFacade.ts b/packages/stryker/src/MutatorFacade.ts index bd0bf3a8eb..c989d615b5 100644 --- a/packages/stryker/src/MutatorFacade.ts +++ b/packages/stryker/src/MutatorFacade.ts @@ -10,7 +10,7 @@ export default class MutatorFacade implements Mutator { constructor(private config: Config) { } - mutate(inputFiles: File[]): Mutant[] { + mutate(inputFiles: ReadonlyArray): ReadonlyArray { return MutatorFactory.instance() .create(this.getMutatorName(this.config.mutator), this.config) .mutate(inputFiles); diff --git a/packages/stryker/src/Sandbox.ts b/packages/stryker/src/Sandbox.ts index 372748f822..82c410cbb4 100644 --- a/packages/stryker/src/Sandbox.ts +++ b/packages/stryker/src/Sandbox.ts @@ -3,14 +3,14 @@ import * as path from 'path'; import { getLogger } from 'log4js'; import * as mkdirp from 'mkdirp'; import { RunResult } from 'stryker-api/test_runner'; -import { File, FileKind, FileDescriptor } from 'stryker-api/core'; +import { File } from 'stryker-api/core'; import { TestFramework } from 'stryker-api/test_framework'; import { wrapInClosure } from './utils/objectUtils'; import TestRunnerDecorator from './isolated-runner/TestRunnerDecorator'; import ResilientTestRunnerFactory from './isolated-runner/ResilientTestRunnerFactory'; import IsolatedRunnerOptions from './isolated-runner/IsolatedRunnerOptions'; import { TempFolder } from './utils/TempFolder'; -import * as fileUtils from './utils/fileUtils'; +import { writeFile } from './utils/fileUtils'; import TestableMutant, { TestSelectionResult } from './TestableMutant'; import TranspiledMutant from './TranspiledMutant'; @@ -25,23 +25,11 @@ export default class Sandbox { private fileMap: FileMap; private files: File[]; private workingFolder: string; - private testHooksFile = path.resolve('___testHooksForStryker.js'); private constructor(private options: Config, private index: number, files: ReadonlyArray, private testFramework: TestFramework | null) { this.workingFolder = TempFolder.instance().createRandomFolder('sandbox'); this.log.debug('Creating a sandbox for files in %s', this.workingFolder); this.files = files.slice(); // Create a copy - if (testFramework) { - this.testHooksFile = path.resolve('___testHooksForStryker.js'); - this.files.unshift({ - name: this.testHooksFile, - content: '', - mutated: false, - included: true, - transpiled: false, - kind: FileKind.Text - }); - } } private async initialize(): Promise { @@ -55,8 +43,8 @@ export default class Sandbox { return sandbox.initialize().then(() => sandbox); } - public run(timeout: number): Promise { - return this.testRunner.run({ timeout }); + public run(timeout: number, testHooks: string | undefined): Promise { + return this.testRunner.run({ timeout, testHooks }); } public dispose(): Promise { @@ -68,32 +56,21 @@ export default class Sandbox { if (transpiledMutant.mutant.testSelectionResult === TestSelectionResult.Failed) { this.log.warn(`Failed find coverage data for this mutant, running all tests. This might have an impact on performance: ${transpiledMutant.mutant.toString()}`); } - await Promise.all(mutantFiles.map(mutatedFile => this.writeFileInSandbox(mutatedFile)).concat(this.filterTests(transpiledMutant.mutant))); - const runResult = await this.run(this.calculateTimeout(transpiledMutant.mutant)); + await Promise.all(mutantFiles.map(mutatedFile => this.writeFileInSandbox(mutatedFile))); + const runResult = await this.run(this.calculateTimeout(transpiledMutant.mutant), this.getFilterTestsHooks(transpiledMutant.mutant)); await this.reset(mutantFiles); return runResult; } - private reset(mutatedFiles: File[]) { + private reset(mutatedFiles: ReadonlyArray) { const originalFiles = this.files.filter(originalFile => mutatedFiles.some(mutatedFile => mutatedFile.name === originalFile.name)); - return Promise.all(originalFiles.map(file => { - if (file.kind !== FileKind.Web) { - return fileUtils.writeFile(this.fileMap[file.name], file.content); - } else { - return Promise.resolve(); - } - })); + return Promise.all(originalFiles.map(file => writeFile(this.fileMap[file.name], file.content))); } private writeFileInSandbox(file: File): Promise { - switch (file.kind) { - case FileKind.Web: - return Promise.resolve(); - default: - const fileNameInSandbox = this.fileMap[file.name]; - return fileUtils.writeFile(fileNameInSandbox, file.content); - } + const fileNameInSandbox = this.fileMap[file.name]; + return writeFile(fileNameInSandbox, file.content); } private fillSandbox(): Promise { @@ -104,31 +81,17 @@ export default class Sandbox { } private fillFile(file: File): Promise { - switch (file.kind) { - case FileKind.Web: - this.fileMap[file.name] = file.name; - return Promise.resolve(); - default: - const cwd = process.cwd(); - const relativePath = path.relative(cwd, file.name); - const folderName = path.join(this.workingFolder, path.dirname(relativePath)); - mkdirp.sync(folderName); - const targetFile = path.join(folderName, path.basename(relativePath)); - this.fileMap[file.name] = targetFile; - return fileUtils.writeFile(targetFile, file.content); - } + const relativePath = path.relative(process.cwd(), file.name); + const folderName = path.join(this.workingFolder, path.dirname(relativePath)); + mkdirp.sync(folderName); + const targetFile = path.join(folderName, path.basename(relativePath)); + this.fileMap[file.name] = targetFile; + return writeFile(targetFile, file.content); } private initializeTestRunner(): void | Promise { - const files: FileDescriptor[] = this.files.map(originalFile => ({ - name: this.fileMap[originalFile.name], - mutated: originalFile.mutated, - included: originalFile.included, - kind: originalFile.kind, - transpiled: originalFile.transpiled - })); const settings: IsolatedRunnerOptions = { - files, + fileNames: Object.keys(this.fileMap).map(sourceFileName => this.fileMap[sourceFileName]), strykerOptions: this.options, port: this.options.port + this.index, sandboxWorkingFolder: this.workingFolder @@ -143,12 +106,11 @@ export default class Sandbox { return (this.options.timeoutFactor * baseTimeout) + this.options.timeoutMs; } - private filterTests(mutant: TestableMutant) { + private getFilterTestsHooks(mutant: TestableMutant): string | undefined { if (this.testFramework) { - let fileContent = wrapInClosure(this.testFramework.filter(mutant.selectedTests)); - return fileUtils.writeFile(this.fileMap[this.testHooksFile], fileContent); + return wrapInClosure(this.testFramework.filter(mutant.selectedTests)); } else { - return Promise.resolve(void 0); + return undefined; } } } \ No newline at end of file diff --git a/packages/stryker/src/SandboxPool.ts b/packages/stryker/src/SandboxPool.ts index 4383ff7d27..0308040b9c 100644 --- a/packages/stryker/src/SandboxPool.ts +++ b/packages/stryker/src/SandboxPool.ts @@ -12,7 +12,7 @@ export default class SandboxPool { private readonly sandboxes: Sandbox[] = []; private isDisposed: boolean = false; - constructor(private options: Config, private testFramework: TestFramework | null, private initialFiles: File[]) { + constructor(private options: Config, private testFramework: TestFramework | null, private initialFiles: ReadonlyArray) { } public streamSandboxes(): Observable { diff --git a/packages/stryker/src/SourceFile.ts b/packages/stryker/src/SourceFile.ts index af38e95efb..7814162f46 100644 --- a/packages/stryker/src/SourceFile.ts +++ b/packages/stryker/src/SourceFile.ts @@ -1,4 +1,4 @@ -import { TextFile, Range, Location, Position } from 'stryker-api/core'; +import { File, Range, Location, Position } from 'stryker-api/core'; const enum CharacterCodes { maxAsciiCharacter = 0x7F, @@ -30,7 +30,7 @@ export default class SourceFile { private lineStarts: number[]; - constructor(public file: TextFile) { + constructor(public file: File) { this.lineStarts = this.computeLineStarts(); } @@ -38,8 +38,8 @@ export default class SourceFile { return this.file.name; } - get content() { - return this.file.content; + get content(): string { + return this.file.textContent; } getLocation(range: Range): Location { @@ -99,12 +99,12 @@ export default class SourceFile { const result: number[] = []; let pos = 0; let lineStart = 0; - while (pos < this.file.content.length) { - const ch = this.file.content.charCodeAt(pos); + while (pos < this.file.textContent.length) { + const ch = this.file.textContent.charCodeAt(pos); pos++; switch (ch) { case CharacterCodes.carriageReturn: - if (this.file.content.charCodeAt(pos) === CharacterCodes.lineFeed) { + if (this.file.textContent.charCodeAt(pos) === CharacterCodes.lineFeed) { pos++; } // falls through diff --git a/packages/stryker/src/Stryker.ts b/packages/stryker/src/Stryker.ts index 1aea470923..d8fc1c7b61 100644 --- a/packages/stryker/src/Stryker.ts +++ b/packages/stryker/src/Stryker.ts @@ -1,12 +1,12 @@ import { Config, ConfigEditorFactory } from 'stryker-api/config'; -import { StrykerOptions, MutatorDescriptor, File } from 'stryker-api/core'; +import { StrykerOptions, MutatorDescriptor } from 'stryker-api/core'; import { MutantResult } from 'stryker-api/report'; import { TestFramework } from 'stryker-api/test_framework'; import { Mutant } from 'stryker-api/mutant'; import ReporterOrchestrator from './ReporterOrchestrator'; import TestFrameworkOrchestrator from './TestFrameworkOrchestrator'; import MutantTestMatcher from './MutantTestMatcher'; -import InputFileResolver from './InputFileResolver'; +import InputFileResolver from './input/InputFileResolver'; import ConfigReader from './ConfigReader'; import PluginLoader from './PluginLoader'; import ScoreResultCalculator from './ScoreResultCalculator'; @@ -19,7 +19,7 @@ import StrictReporter from './reporters/StrictReporter'; import MutatorFacade from './MutatorFacade'; import InitialTestExecutor, { InitialTestRunResult } from './process/InitialTestExecutor'; import MutationTestExecutor from './process/MutationTestExecutor'; -import SourceMapper from './transpiler/SourceMapper'; +import InputFileCollection from './input/InputFileCollection'; export default class Stryker { @@ -50,33 +50,34 @@ export default class Stryker { async runMutationTest(): Promise { this.timer.reset(); const inputFiles = await new InputFileResolver(this.config.mutate, this.config.files, this.reporter).resolve(); - TempFolder.instance().initialize(); - const initialTestRunProcess = this.createInitialTestRunProcess(inputFiles); - const initialTestRunResult = await initialTestRunProcess.run(); - const testableMutants = await this.mutate(inputFiles, initialTestRunResult); - if (initialTestRunResult.runResult.tests.length && testableMutants.length) { - const mutationTestExecutor = this.createMutationTester(inputFiles); - const mutantResults = await mutationTestExecutor.run(testableMutants); - this.reportScore(mutantResults); - await this.wrapUpReporter(); - await TempFolder.instance().clean(); - await this.logDone(); - return mutantResults; - } else { - return Promise.resolve([]); + if (inputFiles.files.length) { + TempFolder.instance().initialize(); + const initialTestRunProcess = new InitialTestExecutor(this.config, inputFiles, this.testFramework, this.timer); + const initialTestRunResult = await initialTestRunProcess.run(); + const testableMutants = await this.mutate(inputFiles, initialTestRunResult); + if (initialTestRunResult.runResult.tests.length && testableMutants.length) { + const mutationTestExecutor = new MutationTestExecutor(this.config, inputFiles.files, this.testFramework, this.reporter); + const mutantResults = await mutationTestExecutor.run(testableMutants); + this.reportScore(mutantResults); + await this.wrapUpReporter(); + await TempFolder.instance().clean(); + await this.logDone(); + return mutantResults; + } } + return Promise.resolve([]); } - private mutate(inputFiles: File[], initialTestRunResult: InitialTestRunResult) { + private mutate(input: InputFileCollection, initialTestRunResult: InitialTestRunResult) { const mutator = new MutatorFacade(this.config); - const allMutants = mutator.mutate(inputFiles); + const allMutants = mutator.mutate(input.filesToMutate); const includedMutants = this.removeExcludedMutants(allMutants); this.logMutantCount(includedMutants.length, allMutants.length); const mutantRunResultMatcher = new MutantTestMatcher( includedMutants, - inputFiles, + input.filesToMutate, initialTestRunResult.runResult, - SourceMapper.create(initialTestRunResult.transpiledFiles, this.config), + initialTestRunResult.sourceMapper, initialTestRunResult.coverageMaps, this.config, this.reporter); @@ -97,7 +98,7 @@ export default class Stryker { this.log.info(mutantCountMessage); } - private removeExcludedMutants(mutants: Mutant[]): Mutant[] { + private removeExcludedMutants(mutants: ReadonlyArray): ReadonlyArray { if (typeof this.config.mutator === 'string') { return mutants; } else { @@ -128,7 +129,14 @@ export default class Stryker { } private freezeConfig() { - freezeRecursively(this.config); + // A config class instance is not serializable using surrial. + // This is a temporary work around + // See https://github.com/stryker-mutator/stryker/issues/365 + const config: Config = {} as any; + for (let prop in this.config) { + config[prop] = this.config[prop]; + } + this.config = freezeRecursively(config); if (this.log.isDebugEnabled()) { this.log.debug(`Using config: ${JSON.stringify(this.config)}`); } @@ -142,14 +150,6 @@ export default class Stryker { log4js.setGlobalLogLevel(this.config.logLevel); } - private createMutationTester(inputFiles: File[]) { - return new MutationTestExecutor(this.config, inputFiles, this.testFramework, this.reporter); - } - - private createInitialTestRunProcess(inputFiles: File[]) { - return new InitialTestExecutor(this.config, inputFiles, this.testFramework, this.timer); - } - private reportScore(mutantResults: MutantResult[]) { const calculator = new ScoreResultCalculator(); const score = calculator.calculate(mutantResults); diff --git a/packages/stryker/src/TestableMutant.ts b/packages/stryker/src/TestableMutant.ts index 70c4dd358a..bab3bd50f2 100644 --- a/packages/stryker/src/TestableMutant.ts +++ b/packages/stryker/src/TestableMutant.ts @@ -8,7 +8,7 @@ import { TestSelection } from 'stryker-api/test_framework'; export enum TestSelectionResult { Failed, - FailedButAlreadyReporter, + FailedButAlreadyReported, Success } @@ -32,10 +32,6 @@ export default class TestableMutant { return this.mutant.fileName; } - get included() { - return this.sourceFile.file.included; - } - get mutatorName() { return this.mutant.mutatorName; } diff --git a/packages/stryker/src/TranspiledMutant.ts b/packages/stryker/src/TranspiledMutant.ts index 0d96b31069..d80349f565 100644 --- a/packages/stryker/src/TranspiledMutant.ts +++ b/packages/stryker/src/TranspiledMutant.ts @@ -1,6 +1,5 @@ import TestableMutant from './TestableMutant'; -import { TranspileResult } from 'stryker-api/transpile'; -import { File, TextFile } from 'stryker-api/core'; +import TranspileResult from './transpiler/TranspileResult'; export default class TranspiledMutant { @@ -11,21 +10,4 @@ export default class TranspiledMutant { * @param changedAnyTranspiledFiles Indicated whether or not this mutant changed the transpiled output files. This is not always the case, for example: mutating a TS interface */ constructor(public mutant: TestableMutant, public transpileResult: TranspileResult, public changedAnyTranspiledFiles: boolean) { } - - static create(mutant: TestableMutant, transpileResult: TranspileResult, unMutatedFiles: File[]) { - return new TranspiledMutant(mutant, transpileResult, someFilesChanged()); - - function someFilesChanged(): boolean { - return transpileResult.outputFiles.some(file => fileChanged(file)); - } - - function fileChanged(file: File) { - if (unMutatedFiles) { - const unMutatedFile = unMutatedFiles.find(f => f.name === file.name); - return !unMutatedFile || (unMutatedFile as TextFile).content !== (file as TextFile).content; - } else { - return true; - } - } - } } \ No newline at end of file diff --git a/packages/stryker/src/child-proxy/ChildProcessProxy.ts b/packages/stryker/src/child-proxy/ChildProcessProxy.ts index a48423633f..3f254c2a43 100644 --- a/packages/stryker/src/child-proxy/ChildProcessProxy.ts +++ b/packages/stryker/src/child-proxy/ChildProcessProxy.ts @@ -1,7 +1,9 @@ import { fork, ChildProcess } from 'child_process'; -import { WorkerMessage, WorkerMessageKind, ParentMessage, autoStart } from './messageProtocol'; +import { File } from 'stryker-api/core'; +import { WorkerMessage, WorkerMessageKind, ParentMessage, autoStart, ParentMessageKind } from './messageProtocol'; import { serialize, deserialize } from '../utils/objectUtils'; import Task from '../utils/Task'; +import { getLogger } from 'log4js'; export type ChildProxy = { [K in keyof T]: (...args: any[]) => Promise; @@ -13,6 +15,7 @@ export default class ChildProcessProxy { private worker: ChildProcess; private initTask: Task; private workerTasks: Task[] = []; + private log = getLogger(ChildProcessProxy.name); private constructor(requirePath: string, logLevel: string, plugins: string[], private constructorFunction: { new(...params: any[]): T }, constructorParams: any[]) { this.worker = fork(require.resolve('./ChildProcessProxyWorker'), [autoStart], { silent: false, execArgv: [] }); @@ -71,11 +74,20 @@ export default class ChildProcessProxy { private listenToWorkerMessages() { this.worker.on('message', (serializedMessage: string) => { - const message: ParentMessage = deserialize(serializedMessage); - if (message === 'init_done') { - this.initTask.resolve(undefined); - } else { - this.workerTasks[message.correlationId].resolve(message.result); + const message: ParentMessage = deserialize(serializedMessage, [File]); + switch (message.kind) { + case ParentMessageKind.Initialized: + this.initTask.resolve(undefined); + break; + case ParentMessageKind.Result: + this.workerTasks[message.correlationId].resolve(message.result); + break; + case ParentMessageKind.Rejection: + this.workerTasks[message.correlationId].reject(new Error(message.error)); + break; + default: + this.logUnidentifiedMessage(message); + break; } }); } @@ -83,4 +95,8 @@ export default class ChildProcessProxy { public dispose() { this.worker.kill(); } + + private logUnidentifiedMessage(message: never) { + this.log.error(`Received unidentified message ${message}`); + } } \ No newline at end of file diff --git a/packages/stryker/src/child-proxy/ChildProcessProxyWorker.ts b/packages/stryker/src/child-proxy/ChildProcessProxyWorker.ts index 7deb5edb1f..db74f4e19e 100644 --- a/packages/stryker/src/child-proxy/ChildProcessProxyWorker.ts +++ b/packages/stryker/src/child-proxy/ChildProcessProxyWorker.ts @@ -1,6 +1,7 @@ -import { serialize, deserialize } from '../utils/objectUtils'; -import { WorkerMessage, WorkerMessageKind, ParentMessage, autoStart } from './messageProtocol'; import { setGlobalLogLevel, getLogger } from 'log4js'; +import { File } from 'stryker-api/core'; +import { serialize, deserialize, errorToString } from '../utils/objectUtils'; +import { WorkerMessage, WorkerMessageKind, ParentMessage, autoStart, ParentMessageKind } from './messageProtocol'; import PluginLoader from '../PluginLoader'; export default class ChildProcessProxyWorker { @@ -22,24 +23,31 @@ export default class ChildProcessProxyWorker { private listenToParent() { const handler = (serializedMessage: string) => { - const message = deserialize(serializedMessage) as WorkerMessage; + const message = deserialize(serializedMessage, [File]); switch (message.kind) { case WorkerMessageKind.Init: setGlobalLogLevel(message.logLevel); new PluginLoader(message.plugins).load(); const RealSubjectClass = require(message.requirePath).default; this.realSubject = new RealSubjectClass(...message.constructorArgs); - this.send('init_done'); + this.send({ kind: ParentMessageKind.Initialized }); this.removeAnyAdditionalMessageListeners(handler); break; case WorkerMessageKind.Work: - const result = this.realSubject[message.methodName](...message.args); - Promise.resolve(result).then(result => { - this.send({ - correlationId: message.correlationId, - result + new Promise(resolve => resolve(this.realSubject[message.methodName](...message.args))) + .then(result => { + this.send({ + kind: ParentMessageKind.Result, + correlationId: message.correlationId, + result + }); + }).catch(error => { + this.send({ + kind: ParentMessageKind.Rejection, + error: errorToString(error), + correlationId: message.correlationId + }); }); - }); this.removeAnyAdditionalMessageListeners(handler); break; } diff --git a/packages/stryker/src/child-proxy/messageProtocol.ts b/packages/stryker/src/child-proxy/messageProtocol.ts index 683ac7d873..980aba168b 100644 --- a/packages/stryker/src/child-proxy/messageProtocol.ts +++ b/packages/stryker/src/child-proxy/messageProtocol.ts @@ -3,8 +3,14 @@ export enum WorkerMessageKind { 'Work' } +export enum ParentMessageKind { + 'Initialized', + 'Result', + 'Rejection' +} + export type WorkerMessage = InitMessage | WorkMessage; -export type ParentMessage = WorkResult | 'init_done'; +export type ParentMessage = WorkResult | { kind: ParentMessageKind.Initialized} | RejectionResult; // Make this an unlikely command line argument // (prevents incidental start of child process) @@ -19,10 +25,17 @@ export interface InitMessage { } export interface WorkResult { + kind: ParentMessageKind.Result; correlationId: number; result: any; } +export interface RejectionResult { + kind: ParentMessageKind.Rejection; + correlationId: number; + error: string; +} + export interface WorkMessage { correlationId: number; kind: WorkerMessageKind.Work; diff --git a/packages/stryker/src/initializer/StrykerConfigWriter.ts b/packages/stryker/src/initializer/StrykerConfigWriter.ts index 66fcecc661..465803ebb4 100644 --- a/packages/stryker/src/initializer/StrykerConfigWriter.ts +++ b/packages/stryker/src/initializer/StrykerConfigWriter.ts @@ -34,10 +34,6 @@ export default class StrykerConfigWriter { selectedReporters: PromptOption[], additionalPiecesOfConfig: Partial[]): Promise { const configObject: Partial = { - files: [ - { pattern: 'src/**/*.js', mutated: true, included: false }, - 'test/**/*.js' - ], testRunner: selectedTestRunner ? selectedTestRunner.name : '', mutator: selectedMutator ? selectedMutator.name : '', transpilers: selectedTranspilers ? selectedTranspilers.map(t => t.name) : [], diff --git a/packages/stryker/src/initializer/StrykerInitializer.ts b/packages/stryker/src/initializer/StrykerInitializer.ts index d1a5d6d68a..c0eed01366 100644 --- a/packages/stryker/src/initializer/StrykerInitializer.ts +++ b/packages/stryker/src/initializer/StrykerInitializer.ts @@ -27,18 +27,18 @@ export default class StrykerInitializer { const selectedTranspilers = await this.selectTranspilers(); const selectedReporters = await this.selectReporters(); const npmDependencies = this.getSelectedNpmDependencies( - [selectedTestRunner, selectedTestFramework, selectedMutator] + [selectedTestRunner, selectedTestFramework, selectedMutator] .concat(selectedTranspilers) .concat(selectedReporters) - ); - this.installNpmDependencies(npmDependencies); + ); await configWriter.write(selectedTestRunner, selectedTestFramework, selectedMutator, selectedTranspilers, selectedReporters, await this.fetchAdditionalConfig(npmDependencies)); - this.out('Done configuring stryker. Please review `stryker.conf.js`, you might need to configure your files and test runner correctly.'); + this.installNpmDependencies(npmDependencies); + this.out('Done configuring stryker. Please review `stryker.conf.js`, you might need to configure transpilers or your test runner correctly.'); this.out('Let\'s kill some mutants with this command: `stryker run`'); } @@ -73,7 +73,7 @@ export default class StrykerInitializer { reporterOptions.push({ name: 'clear-text', npm: null - }, { + }, { name: 'progress', npm: null }, { diff --git a/packages/stryker/src/input/InputFileCollection.ts b/packages/stryker/src/input/InputFileCollection.ts new file mode 100644 index 0000000000..c5ca416cc9 --- /dev/null +++ b/packages/stryker/src/input/InputFileCollection.ts @@ -0,0 +1,31 @@ +import { File } from 'stryker-api/core'; +import { Logger } from 'log4js'; +import { normalizeWhiteSpaces } from '../utils/objectUtils'; + +export default class InputFileCollection { + public readonly files: ReadonlyArray; + public readonly filesToMutate: ReadonlyArray; + + constructor(files: ReadonlyArray, mutateGlobResult: ReadonlyArray) { + this.files = files; + this.filesToMutate = files.filter(file => mutateGlobResult.some(name => name === file.name)); + } + + logFiles(log: Logger) { + if (!this.files.length) { + log.warn(normalizeWhiteSpaces(` + No files selected. Please make sure you either run stryker a git repository context (and don't specify \`files\` in your stryker.conf.js file), + or specify the \`files\` property in your stryker config.`)); + } else { + if (this.filesToMutate.length) { + log.info(`Found ${this.filesToMutate.length} of ${this.files.length} file(s) to be mutated.`); + } else { + log.warn(`No files marked to be mutated, stryker will perform a dry-run without actually mutating anything.`); + } + if (log.isDebugEnabled) { + log.debug(`All input files: ${JSON.stringify(this.files.map(file => file.name), null, 2)}`); + log.debug(`Files to mutate: ${JSON.stringify(this.filesToMutate.map(file => file.name), null, 2)}`); + } + } + } +} diff --git a/packages/stryker/src/input/InputFileResolver.ts b/packages/stryker/src/input/InputFileResolver.ts new file mode 100644 index 0000000000..d4f2f8d9b5 --- /dev/null +++ b/packages/stryker/src/input/InputFileResolver.ts @@ -0,0 +1,182 @@ +import * as path from 'path'; +import * as fs from 'mz/fs'; +import { exec } from 'mz/child_process'; +import { getLogger } from 'log4js'; +import { File } from 'stryker-api/core'; +import { glob } from '../utils/fileUtils'; +import StrictReporter from '../reporters/StrictReporter'; +import { SourceFile } from 'stryker-api/report'; +import StrykerError from '../utils/StrykerError'; +import InputFileCollection from './InputFileCollection'; +import { normalizeWhiteSpaces, filterEmpty, isErrnoException } from '../utils/objectUtils'; + +function toReportSourceFile(file: File): SourceFile { + return { + path: file.name, + content: file.textContent + }; +} + +export default class InputFileResolver { + + private readonly log = getLogger(InputFileResolver.name); + private fileResolver: PatternResolver | undefined; + private mutateResolver: PatternResolver; + + constructor(mutate: string[], files: string[] | undefined, private reporter: StrictReporter) { + this.mutateResolver = PatternResolver.parse(mutate || []); + if (files) { + this.fileResolver = PatternResolver.parse(files); + } + } + + public async resolve(): Promise { + const [inputFileNames, mutateFiles] = await Promise.all([this.resolveInputFiles(), this.mutateResolver.resolve()]); + const files: File[] = await this.readFiles(inputFileNames); + const inputFileCollection = new InputFileCollection(files, mutateFiles); + this.reportAllSourceFilesRead(files); + inputFileCollection.logFiles(this.log); + return inputFileCollection; + } + + private resolveInputFiles() { + if (this.fileResolver) { + return this.fileResolver.resolve(); + } else { + return this.resolveFilesUsingGit(); + } + } + + private resolveFilesUsingGit(): Promise { + return exec('git ls-files --others --exclude-standard --cached', { maxBuffer: 10 * 1000 * 1024 }) + .then(([stdout]) => stdout.toString()) + .then(output => output.split('\n').map(fileName => fileName.trim())) + .then(fileNames => fileNames.filter(fileName => fileName).map(fileName => path.resolve(fileName))) + .catch(error => { + throw new StrykerError(`Cannot determine input files. Either specify a \`files\` array in your stryker configuration, or make sure "${process.cwd()}" is located inside a git repository`, error); + }); + } + + private reportAllSourceFilesRead(allFiles: File[]) { + this.reporter.onAllSourceFilesRead(allFiles.map(toReportSourceFile)); + } + + private reportSourceFilesRead(textFile: File) { + this.reporter.onSourceFileRead(toReportSourceFile(textFile)); + } + + private readFiles(files: string[]): Promise { + return Promise.all(files.map(fileName => this.readFile(fileName))) + .then(filterEmpty); + } + + private readFile(fileName: string): Promise { + return fs.readFile(fileName).then(content => new File(fileName, content)) + .then(file => { + this.reportSourceFilesRead(file); + return file; + }).catch(error => { + if (isErrnoException(error) && error.code === 'ENOENT') { + return null; // file is deleted. This can be a valid result of the git command + } else { + // Rethrow + throw error; + } + }); + } +} + +class PatternResolver { + static normalize(inputFileExpressions: (string | { pattern: string })[]): string[] { + const inputFileDescriptorObjects: { pattern: string }[] = []; + const globExpressions = inputFileExpressions.map(expression => { + if (typeof expression === 'string') { + return expression; + } else { + inputFileDescriptorObjects.push(expression); + return expression.pattern; + } + }); + if (inputFileDescriptorObjects.length) { + new PatternResolver('').log.warn(normalizeWhiteSpaces(` + DEPRECATED: Using the \`InputFileDescriptor\` syntax to + select files is no longer supported. We'll assume: ${JSON.stringify(inputFileDescriptorObjects)} can be migrated + to ${JSON.stringify(inputFileDescriptorObjects.map(_ => _.pattern))} for this mutation run. + Please move any files to mutate into the \`mutate\` array (top level stryker option). + You can fix this warning in 2 ways: + 1) If your project is under git version control, you can remove the "files" patterns all together. + Stryker can figure it out for you. + 2) If your project is not under git version control or you need ignored files in your sandbox, you can replace the + \`InputFileDescriptor\` syntax with strings (as done for this test run).`)); + } + return globExpressions; + } + private readonly log = getLogger(InputFileResolver.name); + private ignore = false; + private globExpression: string; + + constructor(globExpression: string, private previous?: PatternResolver) { + this.ignore = globExpression.indexOf('!') === 0; + if (this.ignore) { + this.globExpression = globExpression.substring(1); + } else { + this.globExpression = globExpression; + } + } + + async resolve(): Promise { + // When the first expression starts with an '!', we skip that one + if (this.ignore && !this.previous) { + return Promise.resolve([]); + } else { + // Start the globbing task for the current descriptor + const globbingTask = this.resolveGlobbingExpression(this.globExpression); + + // If there is a previous globbing expression, resolve that one as well + if (this.previous) { + const results = await Promise.all([this.previous.resolve(), globbingTask]); + const previousFiles = results[0]; + const currentFiles = results[1]; + // If this expression started with a '!', exclude current files + if (this.ignore) { + return previousFiles.filter(previousFile => currentFiles.every(currentFile => previousFile !== currentFile)); + } else { + // Only add files which were not already added + return previousFiles.concat(currentFiles.filter(currentFile => !previousFiles.some(file => file === currentFile))); + } + } else { + return globbingTask; + } + } + } + + static empty(): PatternResolver { + const emptyResolver = new PatternResolver(''); + emptyResolver.ignore = true; + return emptyResolver; + } + + static parse(inputFileExpressions: string[]): PatternResolver { + const expressions = this.normalize(inputFileExpressions); + let current = PatternResolver.empty(); + let expression = expressions.shift(); + while (expression) { + current = new PatternResolver(expression, current); + expression = expressions.shift(); + } + return current; + } + + private async resolveGlobbingExpression(pattern: string): Promise { + let files = await glob(pattern); + if (files.length === 0) { + this.reportEmptyGlobbingExpression(pattern); + } + return files.map((f) => path.resolve(f)); + } + + private reportEmptyGlobbingExpression(expression: string) { + this.log.warn(`Globbing expression "${expression}" did not result in any files.`); + } + +} \ No newline at end of file diff --git a/packages/stryker/src/isolated-runner/IsolatedTestRunnerAdapter.ts b/packages/stryker/src/isolated-runner/IsolatedTestRunnerAdapter.ts index 4376261e42..e1ab9796db 100644 --- a/packages/stryker/src/isolated-runner/IsolatedTestRunnerAdapter.ts +++ b/packages/stryker/src/isolated-runner/IsolatedTestRunnerAdapter.ts @@ -81,7 +81,11 @@ export default class TestRunnerChildProcessAdapter extends EventEmitter implemen break; case 'initDone': if (this.currentTask.kind === 'init') { - this.currentTask.resolve(undefined); + if (message.errorMessage) { + this.currentTask.reject(message.errorMessage); + } else { + this.currentTask.resolve(undefined); + } } else { this.logReceivedUnexpectedMessageWarning(message); } diff --git a/packages/stryker/src/isolated-runner/IsolatedTestRunnerAdapterWorker.ts b/packages/stryker/src/isolated-runner/IsolatedTestRunnerAdapterWorker.ts index 15bc0a3e62..50d2d66d7d 100644 --- a/packages/stryker/src/isolated-runner/IsolatedTestRunnerAdapterWorker.ts +++ b/packages/stryker/src/isolated-runner/IsolatedTestRunnerAdapterWorker.ts @@ -1,7 +1,7 @@ -import { AdapterMessage, RunMessage, StartMessage, EmptyWorkerMessage, WorkerMessage } from './MessageProtocol'; +import { AdapterMessage, RunMessage, StartMessage, WorkerMessage, InitDoneMessage } from './MessageProtocol'; import { TestRunner, RunStatus, TestRunnerFactory, RunResult } from 'stryker-api/test_runner'; import PluginLoader from '../PluginLoader'; -import { getLogger} from 'log4js'; +import { getLogger } from 'log4js'; import { deserialize, errorToString } from '../utils/objectUtils'; class IsolatedTestRunnerAdapterWorker { @@ -16,7 +16,7 @@ class IsolatedTestRunnerAdapterWorker { private listenToMessages() { process.on('message', (serializedMessage: string) => { - const message: AdapterMessage = deserialize(serializedMessage); + const message = deserialize(serializedMessage); switch (message.kind) { case 'start': this.start(message); @@ -66,13 +66,17 @@ class IsolatedTestRunnerAdapterWorker { async init() { if (this.underlyingTestRunner.init) { - await this.underlyingTestRunner.init(); + try { + await this.underlyingTestRunner.init(); + } catch (err) { + this.sendInitDone(errorToString(err)); + } } this.sendInitDone(); } - sendInitDone() { - const message: EmptyWorkerMessage = { kind: 'initDone' }; + sendInitDone(errorMessage: string | null = null) { + const message: InitDoneMessage = { kind: 'initDone', errorMessage }; if (process.send) { process.send(message); } diff --git a/packages/stryker/src/isolated-runner/MessageProtocol.ts b/packages/stryker/src/isolated-runner/MessageProtocol.ts index ebc611bdf1..5cbaa81984 100644 --- a/packages/stryker/src/isolated-runner/MessageProtocol.ts +++ b/packages/stryker/src/isolated-runner/MessageProtocol.ts @@ -3,7 +3,7 @@ import { RunOptions } from 'stryker-api/test_runner'; import IsolatedRunnerOptions from './IsolatedRunnerOptions'; export type AdapterMessage = RunMessage | StartMessage | EmptyAdapterMessage; -export type WorkerMessage = ResultMessage | EmptyWorkerMessage; +export type WorkerMessage = ResultMessage | EmptyWorkerMessage | InitDoneMessage; export interface ResultMessage { kind: 'result'; @@ -21,10 +21,15 @@ export interface StartMessage { runnerOptions: IsolatedRunnerOptions; } +export interface InitDoneMessage { + kind: 'initDone'; + errorMessage: string | null; +} + export interface EmptyAdapterMessage { kind: 'init' | 'dispose'; } export interface EmptyWorkerMessage { - kind: 'initDone' | 'disposeDone'; + kind: 'disposeDone'; } \ No newline at end of file diff --git a/packages/stryker/src/isolated-runner/TestRunnerDecorator.ts b/packages/stryker/src/isolated-runner/TestRunnerDecorator.ts index c5aff7a5cd..c6b61eff26 100644 --- a/packages/stryker/src/isolated-runner/TestRunnerDecorator.ts +++ b/packages/stryker/src/isolated-runner/TestRunnerDecorator.ts @@ -1,11 +1,9 @@ -import { EventEmitter } from 'events'; import { TestRunner, RunOptions, RunResult } from 'stryker-api/test_runner'; -export default class TestRunnerDecorator extends EventEmitter implements TestRunner { +export default class TestRunnerDecorator implements TestRunner { protected innerRunner: TestRunner; constructor(private testRunnerProducer: () => TestRunner) { - super(); this.createInnerRunner(); } diff --git a/packages/stryker/src/isolated-runner/TimeoutDecorator.ts b/packages/stryker/src/isolated-runner/TimeoutDecorator.ts index af9e81f9bc..7bcc666d03 100644 --- a/packages/stryker/src/isolated-runner/TimeoutDecorator.ts +++ b/packages/stryker/src/isolated-runner/TimeoutDecorator.ts @@ -2,6 +2,7 @@ import { RunOptions, RunResult, RunStatus } from 'stryker-api/test_runner'; import { isPromise } from '../utils/objectUtils'; import Task from '../utils/Task'; import TestRunnerDecorator from './TestRunnerDecorator'; +import { getLogger } from 'log4js'; const MAX_WAIT_FOR_DISPOSE = 2500; @@ -10,7 +11,10 @@ const MAX_WAIT_FOR_DISPOSE = 2500; */ export default class TimeoutDecorator extends TestRunnerDecorator { + private readonly log = getLogger(TimeoutDecorator.name); + run(options: RunOptions): Promise { + this.log.debug('Starting timeout timer (%s ms) for a test run', options.timeout); const runTask = new Task(options.timeout, () => this.handleTimeout()); runTask.chainTo(super.run(options)); return runTask.promise; diff --git a/packages/stryker/src/mutators/ES5Mutator.ts b/packages/stryker/src/mutators/ES5Mutator.ts index 7497849ad0..61ebf2d21b 100644 --- a/packages/stryker/src/mutators/ES5Mutator.ts +++ b/packages/stryker/src/mutators/ES5Mutator.ts @@ -1,7 +1,7 @@ import * as _ from 'lodash'; import { Logger, getLogger } from 'log4js'; import { Config } from 'stryker-api/config'; -import { File, TextFile, FileKind } from 'stryker-api/core'; +import { File } from 'stryker-api/core'; import { Mutator, Mutant } from 'stryker-api/mutant'; import * as parserUtils from '../utils/parserUtils'; import { copy } from '../utils/objectUtils'; @@ -35,22 +35,16 @@ export default class ES5Mutator implements Mutator { mutate(files: File[]): Mutant[] { - return _.flatMap(files, file => { - if (file.mutated && file.kind === FileKind.Text) { - return this.mutateForFile(file); - } else { - return []; - } - }); + return _.flatMap(files, file => this.mutateForFile(file)); } - private mutateForFile(file: TextFile): Mutant[] { - const abstractSyntaxTree = parserUtils.parse(file.content); + private mutateForFile(file: File): Mutant[] { + const abstractSyntaxTree = parserUtils.parse(file.textContent); const nodes = new parserUtils.NodeIdentifier().identifyAndFreeze(abstractSyntaxTree); return this.mutateForNodes(file, nodes); } - private mutateForNodes(sourceFile: TextFile, nodes: any[]): Mutant[] { + private mutateForNodes(file: File, nodes: any[]): Mutant[] { return _.flatMap(nodes, astNode => { if (astNode.type) { Object.freeze(astNode); @@ -62,12 +56,12 @@ export default class ES5Mutator implements Mutator { mutatedNodes = [mutatedNodes]; } if (mutatedNodes.length > 0) { - this.log.debug(`The mutator '${mutator.name}' mutated ${mutatedNodes.length} node${mutatedNodes.length > 1 ? 's' : ''} between (Ln ${astNode.loc.start.line}, Col ${astNode.loc.start.column}) and (Ln ${astNode.loc.end.line}, Col ${astNode.loc.end.column}) in file ${sourceFile.name}`); + this.log.debug(`The mutator '${mutator.name}' mutated ${mutatedNodes.length} node${mutatedNodes.length > 1 ? 's' : ''} between (Ln ${astNode.loc.start.line}, Col ${astNode.loc.start.column}) and (Ln ${astNode.loc.end.line}, Col ${astNode.loc.end.column}) in file ${file.name}`); } return mutatedNodes.map(mutatedNode => { const replacement = parserUtils.generate(mutatedNode); const originalNode = nodes[mutatedNode.nodeID]; - const mutant: Mutant = { mutatorName: mutator.name, fileName: sourceFile.name, replacement, range: originalNode.range }; + const mutant: Mutant = { mutatorName: mutator.name, fileName: file.name, replacement, range: originalNode.range }; return mutant; }); } else { diff --git a/packages/stryker/src/process/InitialTestExecutor.ts b/packages/stryker/src/process/InitialTestExecutor.ts index 72b18a8234..887914f5b8 100644 --- a/packages/stryker/src/process/InitialTestExecutor.ts +++ b/packages/stryker/src/process/InitialTestExecutor.ts @@ -2,13 +2,16 @@ import { EOL } from 'os'; import { RunStatus, RunResult, TestResult, TestStatus } from 'stryker-api/test_runner'; import { TestFramework } from 'stryker-api/test_framework'; import { Config } from 'stryker-api/config'; -import { TranspileResult, TranspilerOptions, Transpiler } from 'stryker-api/transpile'; +import { TranspilerOptions, Transpiler } from 'stryker-api/transpile'; import { File } from 'stryker-api/core'; import TranspilerFacade from '../transpiler/TranspilerFacade'; import { getLogger } from 'log4js'; import Sandbox from '../Sandbox'; import Timer from '../utils/Timer'; import CoverageInstrumenterTranspiler, { CoverageMapsByFile } from '../transpiler/CoverageInstrumenterTranspiler'; +import InputFileCollection from '../input/InputFileCollection'; +import SourceMapper from '../transpiler/SourceMapper'; +import { coveragePerTestHooks } from '../transpiler/coverageHooks'; // The initial run might take a while. // For example: angular-bootstrap takes up to 45 seconds. @@ -17,7 +20,7 @@ const INITIAL_RUN_TIMEOUT = 60 * 1000 * 5; export interface InitialTestRunResult { runResult: RunResult; - transpiledFiles: File[]; + sourceMapper: SourceMapper; coverageMaps: CoverageMapsByFile; } @@ -25,38 +28,53 @@ export default class InitialTestExecutor { private readonly log = getLogger(InitialTestExecutor.name); - constructor(private options: Config, private files: File[], private testFramework: TestFramework | null, private timer: Timer) { + constructor(private options: Config, private inputFiles: InputFileCollection, private testFramework: TestFramework | null, private timer: Timer) { } async run(): Promise { - if (this.files.length > 0) { - this.log.info(`Starting initial test run. This may take a while.`); - const result = await this.initialRunInSandbox(); - this.validateResult(result.runResult); - return result; - } else { - this.log.info(`No files have been found. Aborting initial test run.`); - return this.createDryRunResult(); - } + + this.log.info(`Starting initial test run. This may take a while.`); + + // Before we can run the tests we transpile the input files. + // Files that are not transpiled should pass through without transpiling + const transpiledFiles = await this.transpileInputFiles(); + + // Now that we have the transpiled files, we create a source mapper so + // we can figure out which files we need to annotate for code coverage + const sourceMapper = SourceMapper.create(transpiledFiles, this.options); + + // Annotate the transpiled files for code coverage. This allows the + // test runner to report code coverage (if `coverageAnalysis` is enabled) + const { coverageMaps, instrumentedFiles } = await this.annotateForCodeCoverage(transpiledFiles, sourceMapper); + this.logTranspileResult(instrumentedFiles); + + const runResult = await this.runInSandbox(instrumentedFiles); + this.validateResult(runResult); + return { + sourceMapper, + runResult, + coverageMaps + }; } - private async initialRunInSandbox(): Promise { - const coverageInstrumenterTranspiler = this.createCoverageInstrumenterTranspiler(); - const transpilerFacade = this.createTranspilerFacade(coverageInstrumenterTranspiler); - const transpileResult = await transpilerFacade.transpile(this.files); - if (transpileResult.error) { - throw new Error(`Could not transpile input files: ${transpileResult.error}`); - } else { - this.logTranspileResult(transpileResult); - const sandbox = await Sandbox.create(this.options, 0, transpileResult.outputFiles, this.testFramework); - const runResult = await sandbox.run(INITIAL_RUN_TIMEOUT); - await sandbox.dispose(); - return { - runResult, - transpiledFiles: transpileResult.outputFiles, - coverageMaps: coverageInstrumenterTranspiler.fileCoverageMaps - }; - } + private async runInSandbox(files: ReadonlyArray): Promise { + const sandbox = await Sandbox.create(this.options, 0, files, this.testFramework); + const runResult = await sandbox.run(INITIAL_RUN_TIMEOUT, this.getCollectCoverageHooksIfNeeded()); + await sandbox.dispose(); + return runResult; + } + + private async transpileInputFiles(): Promise> { + const transpilerFacade = this.createTranspilerFacade(); + return await transpilerFacade.transpile(this.inputFiles.files); + } + + private async annotateForCodeCoverage(files: ReadonlyArray, sourceMapper: SourceMapper) + : Promise<{ instrumentedFiles: ReadonlyArray, coverageMaps: CoverageMapsByFile }> { + const filesToInstrument = this.inputFiles.filesToMutate.map(mutateFile => sourceMapper.transpiledFileNameFor(mutateFile.name)); + const coverageInstrumenterTranspiler = new CoverageInstrumenterTranspiler(this.options, filesToInstrument); + const instrumentedFiles = await coverageInstrumenterTranspiler.transpile(files); + return { coverageMaps: coverageInstrumenterTranspiler.fileCoverageMaps, instrumentedFiles }; } private validateResult(runResult: RunResult): void { @@ -83,42 +101,36 @@ export default class InitialTestExecutor { throw new Error('Something went wrong in the initial test run'); } - private createDryRunResult(): InitialTestRunResult { - return { - runResult: { - status: RunStatus.Complete, - tests: [], - errorMessages: [] - }, - transpiledFiles: [], - coverageMaps: Object.create(null) - }; - } - /** * Creates a facade for the transpile pipeline. * Also includes the coverage instrumenter transpiler, * which is used to instrument for code coverage when needed. */ - private createTranspilerFacade(coverageInstrumenterTranspiler: CoverageInstrumenterTranspiler): Transpiler { + private createTranspilerFacade(): Transpiler { // Let the transpiler produce source maps only if coverage analysis is enabled const transpilerSettings: TranspilerOptions = { config: this.options, produceSourceMaps: this.options.coverageAnalysis !== 'off' }; - return new TranspilerFacade(transpilerSettings, { - name: CoverageInstrumenterTranspiler.name, - transpiler: coverageInstrumenterTranspiler - }); + return new TranspilerFacade(transpilerSettings); } - private createCoverageInstrumenterTranspiler() { - return new CoverageInstrumenterTranspiler({ produceSourceMaps: true, config: this.options }, this.testFramework); + private getCollectCoverageHooksIfNeeded(): string | undefined { + if (this.options.coverageAnalysis === 'perTest') { + if (this.testFramework) { + // Add piece of javascript to collect coverage per test results + this.log.debug(`Adding test hooks for coverageAnalysis "perTest".`); + return coveragePerTestHooks(this.testFramework); + } else { + this.log.warn('Cannot measure coverage results per test, there is no testFramework and thus no way of executing code right before and after each test.'); + } + } + return undefined; } - private logTranspileResult(transpileResult: TranspileResult) { + private logTranspileResult(transpiledFiles: ReadonlyArray) { if (this.options.transpilers.length && this.log.isDebugEnabled()) { - this.log.debug(`Transpiled files in order:${EOL}${transpileResult.outputFiles.map(f => `${f.name} (included: ${f.included})`).join(EOL)}`); + this.log.debug(`Transpiled files: ${JSON.stringify(transpiledFiles.map(f => `${f.name}`), null, 2)}`); } } @@ -153,4 +165,4 @@ export default class InitialTestExecutor { runResult.tests.forEach(test => message += `${EOL}\t${test.name} (${TestStatus[test.status]})`); this.log.error(message); } -} \ No newline at end of file +} diff --git a/packages/stryker/src/process/MutationTestExecutor.ts b/packages/stryker/src/process/MutationTestExecutor.ts index ee7a0f3cec..6712198d98 100644 --- a/packages/stryker/src/process/MutationTestExecutor.ts +++ b/packages/stryker/src/process/MutationTestExecutor.ts @@ -13,13 +13,13 @@ import SandboxPool from '../SandboxPool'; export default class MutationTestExecutor { - constructor(private config: Config, private inputFiles: File[], private testFramework: TestFramework | null, private reporter: StrictReporter) { + constructor(private config: Config, private inputFiles: ReadonlyArray, private testFramework: TestFramework | null, private reporter: StrictReporter) { } async run(allMutants: TestableMutant[]): Promise { const mutantTranspiler = new MutantTranspiler(this.config); - const transpileResult = await mutantTranspiler.initialize(this.inputFiles); - const sandboxPool = new SandboxPool(this.config, this.testFramework, transpileResult.outputFiles); + const transpiledFiles = await mutantTranspiler.initialize(this.inputFiles); + const sandboxPool = new SandboxPool(this.config, this.testFramework, transpiledFiles); const result = await this.runInsideSandboxes( sandboxPool.streamSandboxes(), mutantTranspiler.transpileMutants(allMutants)); diff --git a/packages/stryker/src/transpiler/CoverageInstrumenterTranspiler.ts b/packages/stryker/src/transpiler/CoverageInstrumenterTranspiler.ts index 5e910459de..0efa6fff20 100644 --- a/packages/stryker/src/transpiler/CoverageInstrumenterTranspiler.ts +++ b/packages/stryker/src/transpiler/CoverageInstrumenterTranspiler.ts @@ -1,12 +1,9 @@ -import { Transpiler, TranspileResult, TranspilerOptions } from 'stryker-api/transpile'; -import { File, FileKind, TextFile } from 'stryker-api/core'; +import { Transpiler } from 'stryker-api/transpile'; import { createInstrumenter, Instrumenter } from 'istanbul-lib-instrument'; -import { errorToString, wrapInClosure } from '../utils/objectUtils'; -import { TestFramework } from 'stryker-api/test_framework'; -import { Logger, getLogger } from 'log4js'; +import { StrykerOptions, File } from 'stryker-api/core'; import { FileCoverageData, Range } from 'istanbul-lib-coverage'; - -const COVERAGE_CURRENT_TEST_VARIABLE_NAME = '__strykerCoverageCurrentTest__'; +import { COVERAGE_CURRENT_TEST_VARIABLE_NAME } from './coverageHooks'; +import StrykerError from '../utils/StrykerError'; export interface CoverageMaps { statementMap: { [key: string]: Range }; @@ -21,23 +18,13 @@ export default class CoverageInstrumenterTranspiler implements Transpiler { private instrumenter: Instrumenter; public fileCoverageMaps: CoverageMapsByFile = Object.create(null); - private log: Logger; - constructor(private settings: TranspilerOptions, private testFramework: TestFramework | null) { + constructor(private settings: StrykerOptions, private filesToInstrument: ReadonlyArray) { this.instrumenter = createInstrumenter({ coverageVariable: this.coverageVariable, preserveComments: true }); - this.log = getLogger(CoverageInstrumenterTranspiler.name); } - public transpile(files: File[]): Promise { - try { - const result: TranspileResult = { - outputFiles: files.map(file => this.instrumentFileIfNeeded(file)), - error: null - }; - return Promise.resolve(this.addCollectCoverageFileIfNeeded(result)); - } catch (error) { - return Promise.resolve(this.errorResult(errorToString(error))); - } + public async transpile(files: ReadonlyArray): Promise> { + return files.map(file => this.instrumentFileIfNeeded(file)); } /** @@ -50,7 +37,7 @@ export default class CoverageInstrumenterTranspiler implements Transpiler { * and after each test copy over the value of that current test to the global coverage object __coverage__ */ private get coverageVariable() { - switch (this.settings.config.coverageAnalysis) { + switch (this.settings.coverageAnalysis) { case 'perTest': return COVERAGE_CURRENT_TEST_VARIABLE_NAME; default: @@ -79,29 +66,23 @@ export default class CoverageInstrumenterTranspiler implements Transpiler { return fileCoverage; } + private instrumentFileIfNeeded(file: File) { - if (this.settings.config.coverageAnalysis !== 'off' && file.kind === FileKind.Text && file.mutated) { + if (this.settings.coverageAnalysis !== 'off' && this.filesToInstrument.some(fileName => fileName === file.name)) { return this.instrumentFile(file); } else { return file; } } - private instrumentFile(sourceFile: TextFile): TextFile { + private instrumentFile(sourceFile: File): File { try { - const content = this.instrumenter.instrumentSync(sourceFile.content, sourceFile.name); + const content = this.instrumenter.instrumentSync(sourceFile.textContent, sourceFile.name); const fileCoverage = this.patchRanges(this.instrumenter.lastFileCoverage()); this.fileCoverageMaps[sourceFile.name] = this.retrieveCoverageMaps(fileCoverage); - return { - mutated: sourceFile.mutated, - included: sourceFile.included, - name: sourceFile.name, - transpiled: sourceFile.transpiled, - kind: FileKind.Text, - content - }; + return new File(sourceFile.name, Buffer.from(content)); } catch (error) { - throw new Error(`Could not instrument "${sourceFile.name}" for code coverage. ${errorToString(error)}`); + throw new StrykerError(`Could not instrument "${sourceFile.name}" for code coverage`, error); } } @@ -113,93 +94,4 @@ export default class CoverageInstrumenterTranspiler implements Transpiler { Object.keys(input.fnMap).forEach(key => output.fnMap[key] = input.fnMap[key].loc); return output; } - - private addCollectCoverageFileIfNeeded(result: TranspileResult): TranspileResult { - if (Object.keys(this.fileCoverageMaps).length && this.settings.config.coverageAnalysis === 'perTest') { - if (this.testFramework) { - // Add piece of javascript to collect coverage per test results - const content = this.coveragePerTestFileContent(this.testFramework); - const fileName = '____collectCoveragePerTest____.js'; - result.outputFiles.unshift({ - kind: FileKind.Text, - name: fileName, - included: true, - transpiled: false, - mutated: false, - content - }); - this.log.debug(`Adding test hooks file for coverageAnalysis "perTest": ${fileName}`); - } else { - return this.errorResult('Cannot measure coverage results per test, there is no testFramework and thus no way of executing code right before and after each test.'); - } - } - return result; - } - - private coveragePerTestFileContent(testFramework: TestFramework): string { - return wrapInClosure(` - var id = 0, globalCoverage, coverageResult; - window.__coverage__ = globalCoverage = { deviations: {} }; - ${testFramework.beforeEach(beforeEachFragmentPerTest)} - ${testFramework.afterEach(afterEachFragmentPerTest)} - ${cloneFunctionFragment}; - `); - } - - private errorResult(error: string) { - return { - error, - outputFiles: [] - }; - } } - -const cloneFunctionFragment = ` -function clone(source) { - var result = source; - if (Array.isArray(source)) { - result = []; - source.forEach(function (child, index) { - result[index] = clone(child); - }); - } else if (typeof source == "object") { - result = {}; - for (var i in source) { - result[i] = clone(source[i]); - } - } - return result; -}`; - -const beforeEachFragmentPerTest = ` -if (!globalCoverage.baseline && window.${COVERAGE_CURRENT_TEST_VARIABLE_NAME}) { -globalCoverage.baseline = clone(window.${COVERAGE_CURRENT_TEST_VARIABLE_NAME}); -}`; - -const afterEachFragmentPerTest = ` -globalCoverage.deviations[id] = coverageResult = {}; -id++; -var coveragePerFile = window.${COVERAGE_CURRENT_TEST_VARIABLE_NAME}; -if(coveragePerFile) { -Object.keys(coveragePerFile).forEach(function (file) { - var coverage = coveragePerFile[file]; - var baseline = globalCoverage.baseline[file]; - var fileResult = { s: {}, f: {} }; - var touchedFile = false; - for(var i in coverage.s){ - if(coverage.s[i] !== baseline.s[i]){ - fileResult.s[i] = coverage.s[i]; - touchedFile = true; - } - } - for(var i in coverage.f){ - if(coverage.f[i] !== baseline.f[i]){ - fileResult.f[i] = coverage.f[i]; - touchedFile = true; - } - } - if(touchedFile){ - coverageResult[file] = fileResult; - } -}); -}`; \ No newline at end of file diff --git a/packages/stryker/src/transpiler/MutantTranspiler.ts b/packages/stryker/src/transpiler/MutantTranspiler.ts index f4121cc771..ed691e1853 100644 --- a/packages/stryker/src/transpiler/MutantTranspiler.ts +++ b/packages/stryker/src/transpiler/MutantTranspiler.ts @@ -2,18 +2,20 @@ import { Observable } from 'rxjs'; import { Config } from 'stryker-api/config'; import TranspilerFacade from './TranspilerFacade'; import TestableMutant from '../TestableMutant'; -import { File, TextFile, FileKind } from 'stryker-api/core'; +import { File } from 'stryker-api/core'; import SourceFile from '../SourceFile'; import ChildProcessProxy, { ChildProxy } from '../child-proxy/ChildProcessProxy'; -import { TranspileResult, TranspilerOptions } from 'stryker-api/transpile'; +import { TranspilerOptions } from 'stryker-api/transpile'; import TranspiledMutant from '../TranspiledMutant'; +import TranspileResult from './TranspileResult'; +import { errorToString } from '../utils/objectUtils'; export default class MutantTranspiler { private transpilerChildProcess: ChildProcessProxy | undefined; private proxy: ChildProxy; private currentMutatedFile: SourceFile; - private unMutatedFiles: File[]; + private unMutatedFiles: ReadonlyArray; /** * Creates the mutant transpiler in a child process if one is defined. @@ -36,10 +38,10 @@ export default class MutantTranspiler { } } - initialize(files: File[]): Promise { - return this.proxy.transpile(files).then((transpileResult: TranspileResult) => { - this.unMutatedFiles = transpileResult.outputFiles; - return transpileResult; + initialize(files: ReadonlyArray): Promise> { + return this.proxy.transpile(files).then((transpiledFiles: ReadonlyArray) => { + this.unMutatedFiles = transpiledFiles; + return transpiledFiles; }); } @@ -50,7 +52,7 @@ export default class MutantTranspiler { const mutant = mutants.shift(); if (mutant) { this.transpileMutant(mutant) - .then(transpileResult => observer.next(TranspiledMutant.create(mutant, transpileResult, this.unMutatedFiles))) + .then(transpiledFiles => observer.next(this.createTranspiledMutant(mutant, transpiledFiles, this.unMutatedFiles))) .then(nextMutant) .catch(error => observer.error(error)); } else { @@ -67,21 +69,33 @@ export default class MutantTranspiler { } } + private createTranspiledMutant(mutant: TestableMutant, transpileResult: TranspileResult, unMutatedFiles: ReadonlyArray) { + return new TranspiledMutant(mutant, transpileResult, someFilesChanged()); + + function someFilesChanged(): boolean { + return transpileResult.outputFiles.some(file => fileChanged(file)); + } + + function fileChanged(file: File) { + if (unMutatedFiles) { + const unMutatedFile = unMutatedFiles.find(f => f.name === file.name); + return !unMutatedFile || unMutatedFile.textContent !== file.textContent; + } else { + return true; + } + } + } + private transpileMutant(mutant: TestableMutant): Promise { - const filesToTranspile: TextFile[] = []; - if (this.currentMutatedFile && this.currentMutatedFile.file.name !== mutant.fileName) { + const filesToTranspile: File[] = []; + if (this.currentMutatedFile && this.currentMutatedFile.name !== mutant.fileName) { filesToTranspile.push(this.currentMutatedFile.file); } this.currentMutatedFile = mutant.sourceFile; - const mutatedFile: TextFile = { - name: mutant.fileName, - content: mutant.mutatedCode, - kind: FileKind.Text, - mutated: this.currentMutatedFile.file.mutated, - transpiled: this.currentMutatedFile.file.transpiled, - included: mutant.included - }; + const mutatedFile = new File(mutant.fileName, Buffer.from(mutant.mutatedCode)); filesToTranspile.push(mutatedFile); - return this.proxy.transpile(filesToTranspile); + return this.proxy.transpile(filesToTranspile) + .then((transpiledFiles: ReadonlyArray) => ({ outputFiles: transpiledFiles, error: null })) + .catch(error => ({ outputFiles: [], error: errorToString(error) })); } } \ No newline at end of file diff --git a/packages/stryker/src/transpiler/SourceMapper.ts b/packages/stryker/src/transpiler/SourceMapper.ts index 4c729ecba6..739a535eb7 100644 --- a/packages/stryker/src/transpiler/SourceMapper.ts +++ b/packages/stryker/src/transpiler/SourceMapper.ts @@ -1,25 +1,26 @@ import * as path from 'path'; import { SourceMapConsumer, RawSourceMap } from 'source-map'; -import { File, FileKind, TextFile, Location, Position } from 'stryker-api/core'; +import { File, Location, Position } from 'stryker-api/core'; import { Config } from 'stryker-api/config'; import { base64Decode } from '../utils/objectUtils'; +import { getLogger } from 'log4js'; +import StrykerError from '../utils/StrykerError'; const SOURCE_MAP_URL_REGEX = /\/\/\s*#\s*sourceMappingURL=(.*)/g; - // This file contains source mapping logic. // It reads transpiled output files (*.js) and scans it for comments like these: sourceMappingURL=*.js.map // If it finds it, it will use mozilla's source-map to implement the `transpiledLocationFor` method. - export interface MappedLocation { fileName: string; location: Location; } -export class SourceMapError extends Error { - constructor(message: string) { - super(`${message}. Cannot analyse code coverage. Setting \`coverageAnalysis: "off"\` in your stryker.conf.js will prevent this error, but forces Stryker to run each test for each mutant.`); +export class SourceMapError extends StrykerError { + constructor(message: string, innerError?: Error) { + super(`${message}. Cannot analyse code coverage. Setting \`coverageAnalysis: "off"\` in your stryker.conf.js will prevent this error, but forces Stryker to run each test for each mutant.`, + innerError); Error.captureStackTrace(this, SourceMapError); // TS recommendation: https://github.com/Microsoft/TypeScript/wiki/Breaking-Changes#extending-built-ins-like-error-array-and-map-may-no-longer-work Object.setPrototypeOf(this, SourceMapError.prototype); @@ -38,7 +39,9 @@ export default abstract class SourceMapper { */ abstract transpiledLocationFor(originalLocation: MappedLocation): MappedLocation; - static create(transpiledFiles: File[], config: Config): SourceMapper { + abstract transpiledFileNameFor(originalFileName: string): string; + + static create(transpiledFiles: ReadonlyArray, config: Config): SourceMapper { if (config.transpilers.length && config.coverageAnalysis !== 'off') { return new TranspiledSourceMapper(transpiledFiles); } else { @@ -50,30 +53,35 @@ export default abstract class SourceMapper { export class TranspiledSourceMapper extends SourceMapper { private sourceMaps: SourceMapBySource; + private log = getLogger(SourceMapper.name); - constructor(private transpiledFiles: File[]) { + constructor(private transpiledFiles: ReadonlyArray) { super(); } + /** + * @inheritDoc + */ + transpiledFileNameFor(originalFileName: string) { + const sourceMap = this.getSourceMap(originalFileName); + return sourceMap.transpiledFile.name; + } + /** * @inheritdoc */ transpiledLocationFor(originalLocation: MappedLocation): MappedLocation { const sourceMap = this.getSourceMap(originalLocation.fileName); - if (!sourceMap) { - throw new SourceMapError(`Source map not found for "${originalLocation.fileName}"`); - } else { - const relativeSource = this.getRelativeSource(sourceMap, originalLocation); - const start = sourceMap.generatedPositionFor(originalLocation.location.start, relativeSource); - const end = sourceMap.generatedPositionFor(originalLocation.location.end, relativeSource); - return { - fileName: sourceMap.transpiledFile.name, - location: { - start, - end - } - }; - } + const relativeSource = this.getRelativeSource(sourceMap, originalLocation); + const start = sourceMap.generatedPositionFor(originalLocation.location.start, relativeSource); + const end = sourceMap.generatedPositionFor(originalLocation.location.end, relativeSource); + return { + fileName: sourceMap.transpiledFile.name, + location: { + start, + end + } + }; } private getRelativeSource(from: SourceMap, to: MappedLocation) { @@ -84,11 +92,16 @@ export class TranspiledSourceMapper extends SourceMapper { /** * Gets the source map for given file */ - private getSourceMap(sourceFileName: string): SourceMap | undefined { + private getSourceMap(sourceFileName: string): SourceMap { if (!this.sourceMaps) { this.sourceMaps = this.createSourceMaps(); } - return this.sourceMaps[path.resolve(sourceFileName)]; + const sourceMap: SourceMap | undefined = this.sourceMaps[path.resolve(sourceFileName)]; + if (sourceMap) { + return sourceMap; + } else { + throw new SourceMapError(`Source map not found for "${sourceFileName}"`); + } } /** @@ -97,9 +110,9 @@ export class TranspiledSourceMapper extends SourceMapper { private createSourceMaps(): SourceMapBySource { const sourceMaps: SourceMapBySource = Object.create(null); this.transpiledFiles.forEach(transpiledFile => { - if (transpiledFile.mutated && transpiledFile.kind === FileKind.Text) { - const sourceMapFile = this.getSourceMapForFile(transpiledFile); - const rawSourceMap: RawSourceMap = JSON.parse(sourceMapFile.content); + const sourceMapFile = this.getSourceMapForFile(transpiledFile); + if (sourceMapFile) { + const rawSourceMap = this.getRawSourceMap(sourceMapFile); const sourceMap = new SourceMap(transpiledFile, sourceMapFile.name, rawSourceMap); rawSourceMap.sources.forEach(source => { const sourceFileName = path.resolve(path.dirname(sourceMapFile.name), source); @@ -110,10 +123,21 @@ export class TranspiledSourceMapper extends SourceMapper { return sourceMaps; } - private getSourceMapForFile(transpiledFile: TextFile) { + private getRawSourceMap(sourceMapFile: File): RawSourceMap { + try { + return JSON.parse(sourceMapFile.textContent); + } catch (error) { + throw new SourceMapError(`Source map file "${sourceMapFile.name}" could not be parsed as json`, error); + } + } + + private getSourceMapForFile(transpiledFile: File): File | null { const sourceMappingUrl = this.getSourceMapUrl(transpiledFile); - const sourceMapFile = this.getSourceMapFileFromUrl(sourceMappingUrl, transpiledFile); - return sourceMapFile; + if (sourceMappingUrl) { + return this.getSourceMapFileFromUrl(sourceMappingUrl, transpiledFile); + } else { + return null; + } } /** @@ -121,14 +145,10 @@ export class TranspiledSourceMapper extends SourceMapper { * @param sourceMapUrl The source map url. Can be a data url (data:application/json;base64,ABC...), or an actual file url * @param transpiledFile The transpiled file for which the data url is */ - private getSourceMapFileFromUrl(sourceMapUrl: string, transpiledFile: File): TextFile { + private getSourceMapFileFromUrl(sourceMapUrl: string, transpiledFile: File): File { const sourceMapFile = this.isInlineUrl(sourceMapUrl) ? this.getInlineSourceMap(sourceMapUrl, transpiledFile) : this.getExternalSourceMap(sourceMapUrl, transpiledFile); - if (sourceMapFile.kind === FileKind.Text) { - return sourceMapFile; - } else { - throw new SourceMapError(`Source map file "${sourceMapFile.name}" has the wrong file kind. "${FileKind[sourceMapFile.kind]}" instead of "${FileKind[FileKind.Text]}"`); - } + return sourceMapFile; } private isInlineUrl(sourceMapUrl: string) { @@ -138,18 +158,11 @@ export class TranspiledSourceMapper extends SourceMapper { /** * Gets the source map from a data url */ - private getInlineSourceMap(sourceMapUrl: string, transpiledFile: File): TextFile { + private getInlineSourceMap(sourceMapUrl: string, transpiledFile: File): File { const supportedDataPrefix = 'data:application/json;base64,'; if (sourceMapUrl.startsWith(supportedDataPrefix)) { const content = base64Decode(sourceMapUrl.substr(supportedDataPrefix.length)); - return { - name: transpiledFile.name, - content, - kind: FileKind.Text, - included: false, - mutated: false, - transpiled: false - }; + return new File(transpiledFile.name, content); } else { throw new SourceMapError(`Source map file for "${transpiledFile.name}" cannot be read. Data url "${sourceMapUrl.substr(0, sourceMapUrl.lastIndexOf(','))}" found, where "${supportedDataPrefix.substr(0, supportedDataPrefix.length - 1)}" was expected`); } @@ -171,18 +184,20 @@ export class TranspiledSourceMapper extends SourceMapper { /** * Gets the source map url from a transpiled file (the last comment with sourceMappingURL= ...) */ - private getSourceMapUrl(transpiledFile: TextFile): string { + private getSourceMapUrl(transpiledFile: File): string | null { SOURCE_MAP_URL_REGEX.lastIndex = 0; let currentMatch: RegExpExecArray | null; let lastMatch: RegExpExecArray | null = null; // Retrieve the final sourceMappingURL comment in the file - while (currentMatch = SOURCE_MAP_URL_REGEX.exec(transpiledFile.content)) { + while (currentMatch = SOURCE_MAP_URL_REGEX.exec(transpiledFile.textContent)) { lastMatch = currentMatch; } if (lastMatch) { + this.log.debug('Source map url found in transpiled file "%s"', transpiledFile.name); return lastMatch[1]; } else { - throw new SourceMapError(`No source map reference found in transpiled file "${transpiledFile.name}"`); + this.log.debug('No source map url found in transpiled file "%s"', transpiledFile.name); + return null; } } } @@ -190,6 +205,13 @@ export class TranspiledSourceMapper extends SourceMapper { export class PassThroughSourceMapper extends SourceMapper { + /** + * @inheritdoc + */ + transpiledFileNameFor(originalFileName: string): string { + return originalFileName; + } + /** * @inheritdoc */ @@ -201,7 +223,7 @@ export class PassThroughSourceMapper extends SourceMapper { class SourceMap { private sourceMap: SourceMapConsumer; - constructor(public transpiledFile: TextFile, public sourceMapFileName: string, rawSourceMap: RawSourceMap) { + constructor(public transpiledFile: File, public sourceMapFileName: string, rawSourceMap: RawSourceMap) { this.sourceMap = new SourceMapConsumer(rawSourceMap); } generatedPositionFor(originalPosition: Position, relativeSource: string): Position { @@ -216,6 +238,8 @@ class SourceMap { column: transpiledPosition.column }; } + + } interface SourceMapBySource { diff --git a/packages/stryker/src/transpiler/TranspileResult.ts b/packages/stryker/src/transpiler/TranspileResult.ts new file mode 100644 index 0000000000..7774b4a91c --- /dev/null +++ b/packages/stryker/src/transpiler/TranspileResult.ts @@ -0,0 +1,6 @@ +import { File } from 'stryker-api/core'; + +export default interface TranspileResult { + outputFiles: ReadonlyArray; + error: string | null; +} \ No newline at end of file diff --git a/packages/stryker/src/transpiler/TranspilerFacade.ts b/packages/stryker/src/transpiler/TranspilerFacade.ts index 0bfd598249..65b9fd345a 100644 --- a/packages/stryker/src/transpiler/TranspilerFacade.ts +++ b/packages/stryker/src/transpiler/TranspilerFacade.ts @@ -1,5 +1,6 @@ import { File } from 'stryker-api/core'; -import { Transpiler, TranspileResult, TranspilerOptions, TranspilerFactory } from 'stryker-api/transpile'; +import { Transpiler, TranspilerOptions, TranspilerFactory } from 'stryker-api/transpile'; +import StrykerError from '../utils/StrykerError'; class NamedTranspiler { constructor(public name: string, public transpiler: Transpiler) { } @@ -9,40 +10,29 @@ export default class TranspilerFacade implements Transpiler { private innerTranspilers: NamedTranspiler[]; - constructor(options: TranspilerOptions, additionalTranspiler?: { name: string, transpiler: Transpiler }) { + constructor(options: TranspilerOptions) { this.innerTranspilers = options.config.transpilers .map(transpilerName => new NamedTranspiler(transpilerName, TranspilerFactory.instance().create(transpilerName, options))); - if (additionalTranspiler) { - this.innerTranspilers.push(new NamedTranspiler(additionalTranspiler.name, additionalTranspiler.transpiler)); - } } - public transpile(files: File[]): Promise { - return this.performTranspileChain(this.createPassThruTranspileResult(files)); + public transpile(files: ReadonlyArray): Promise> { + return this.performTranspileChain(files); } private async performTranspileChain( - currentResult: TranspileResult, + input: ReadonlyArray, remainingChain: NamedTranspiler[] = this.innerTranspilers.slice() - ): Promise { - const next = remainingChain.shift(); - if (next) { - const nextResult = await next.transpiler.transpile(currentResult.outputFiles); - if (nextResult.error) { - nextResult.error = `Execute ${next.name}: ${nextResult.error}`; - return nextResult; - } else { - return this.performTranspileChain(nextResult, remainingChain); - } + ): Promise> { + const current = remainingChain.shift(); + if (current) { + const output = await current.transpiler.transpile(input) + .catch(error => { + throw new StrykerError(`An error occurred in transpiler "${current.name}"`, error); + }); + return this.performTranspileChain(output, remainingChain); } else { - return currentResult; + return input; } } - private createPassThruTranspileResult(input: File[]): TranspileResult { - return { - error: null, - outputFiles: input - }; - } } \ No newline at end of file diff --git a/packages/stryker/src/transpiler/coverageHooks.ts b/packages/stryker/src/transpiler/coverageHooks.ts new file mode 100644 index 0000000000..dc3ae15643 --- /dev/null +++ b/packages/stryker/src/transpiler/coverageHooks.ts @@ -0,0 +1,64 @@ +import { wrapInClosure } from '../utils/objectUtils'; +import { TestFramework } from 'stryker-api/test_framework'; + +export const COVERAGE_CURRENT_TEST_VARIABLE_NAME = '__strykerCoverageCurrentTest__'; + +const cloneFunctionFragment = ` +function clone(source) { + var result = source; + if (Array.isArray(source)) { + result = []; + source.forEach(function (child, index) { + result[index] = clone(child); + }); + } else if (typeof source == "object") { + result = {}; + for (var i in source) { + result[i] = clone(source[i]); + } + } + return result; +}`; + +const BEFORE_EACH_FRAGMENT_PER_TEST = ` +if (!globalCoverage.baseline && window.${COVERAGE_CURRENT_TEST_VARIABLE_NAME}) { +globalCoverage.baseline = clone(window.${COVERAGE_CURRENT_TEST_VARIABLE_NAME}); +}`; + +const AFTER_EACH_FRAGMENT_PER_TEST = ` +globalCoverage.deviations[id] = coverageResult = {}; +id++; +var coveragePerFile = window.${COVERAGE_CURRENT_TEST_VARIABLE_NAME}; +if(coveragePerFile) { +Object.keys(coveragePerFile).forEach(function (file) { + var coverage = coveragePerFile[file]; + var baseline = globalCoverage.baseline[file]; + var fileResult = { s: {}, f: {} }; + var touchedFile = false; + for(var i in coverage.s){ + if(coverage.s[i] !== baseline.s[i]){ + fileResult.s[i] = coverage.s[i]; + touchedFile = true; + } + } + for(var i in coverage.f){ + if(coverage.f[i] !== baseline.f[i]){ + fileResult.f[i] = coverage.f[i]; + touchedFile = true; + } + } + if(touchedFile){ + coverageResult[file] = fileResult; + } +}); +}`; + +export function coveragePerTestHooks(testFramework: TestFramework): string { + return wrapInClosure(` + var id = 0, globalCoverage, coverageResult; + window.__coverage__ = globalCoverage = { deviations: {} }; + ${testFramework.beforeEach(BEFORE_EACH_FRAGMENT_PER_TEST)} + ${testFramework.afterEach(AFTER_EACH_FRAGMENT_PER_TEST)} + ${cloneFunctionFragment}; + `); +} diff --git a/packages/stryker/src/utils/StrykerError.ts b/packages/stryker/src/utils/StrykerError.ts new file mode 100644 index 0000000000..20e7972568 --- /dev/null +++ b/packages/stryker/src/utils/StrykerError.ts @@ -0,0 +1,10 @@ +import { errorToString } from './objectUtils'; + +export default class StrykerError extends Error { + constructor(message: string, innerError?: Error) { + super(`${message}${innerError ? `. Inner error: ${errorToString(innerError)}` : ''}`); + Error.captureStackTrace(this, StrykerError); + // TS recommendation: https://github.com/Microsoft/TypeScript/wiki/Breaking-Changes#extending-built-ins-like-error-array-and-map-may-no-longer-work + Object.setPrototypeOf(this, StrykerError.prototype); + } +} \ No newline at end of file diff --git a/packages/stryker/src/utils/fileUtils.ts b/packages/stryker/src/utils/fileUtils.ts index 2f9dd31643..5fd4d3caa8 100644 --- a/packages/stryker/src/utils/fileUtils.ts +++ b/packages/stryker/src/utils/fileUtils.ts @@ -1,9 +1,7 @@ import * as fs from 'mz/fs'; -import * as path from 'path'; import * as nodeGlob from 'glob'; import * as mkdirp from 'mkdirp'; import * as rimraf from 'rimraf'; -import { FileKind } from 'stryker-api/core'; export function glob(expression: string): Promise { return new Promise((resolve, reject) => { @@ -35,23 +33,6 @@ export function importModule(moduleName: string) { require(moduleName); } -export function isOnlineFile(path: string): boolean { - return path.indexOf('http://') === 0 || path.indexOf('https://') === 0; -} - -const binaryExtensions = [ - '.png', - '.jpeg', - '.jpg', - '.zip', - '.tar', - '.gif' // Still more to add -]; - -function isBinaryFile(name: string): boolean { - return binaryExtensions.indexOf(path.extname(name)) > -1; -} - /** * Writes data to a specified file. * @param fileName The path to the file. @@ -65,13 +46,3 @@ export function writeFile(fileName: string, data: string | Buffer): Promise(target: T): T { Object.freeze(target); @@ -18,22 +19,6 @@ export function filterEmpty(input: (T | null | void)[]) { return input.filter(item => item !== undefined && item !== null) as T[]; } - -/** - * Serializes javascript without using `JSON.stringify` (directly), as it does not allow for regexes or functions, etc - */ -export const serialize: (obj: any) => string = require('serialize-javascript'); - -/** - * Deserialize javascript without using `JSON.parse` (directly), as it does not allow for regexes or functions, etc - * (Uses eval instead) - */ -export function deserialize(serializedJavascript: String): any { - // tslint:disable - return eval(`(${serializedJavascript})`); - // tslint:enable -} - export function isErrnoException(error: Error): error is NodeJS.ErrnoException { return typeof (error as NodeJS.ErrnoException).code === 'string'; } @@ -88,4 +73,12 @@ export function setExitCode(n: number) { export function base64Decode(base64EncodedString: string) { return Buffer.from(base64EncodedString, 'base64').toString('utf8'); +} + +/** + * Consolidates multiple consecutive white spaces into a single space. + * @param str The string to be normalized + */ +export function normalizeWhiteSpaces(str: string) { + return str.replace(/\s+/g, ' ').trim(); } \ No newline at end of file diff --git a/packages/stryker/stryker.conf.js b/packages/stryker/stryker.conf.js index 9970d8205b..5ab5a51774 100644 --- a/packages/stryker/stryker.conf.js +++ b/packages/stryker/stryker.conf.js @@ -6,20 +6,24 @@ module.exports = function (config) { if (typescript) { config.set({ files: [ - { pattern: 'package.json', transpiled: false, included: false, mutated: false }, - '!test/integration/**/*.ts', - '!src/**/*.ts', - { pattern: 'src/**/*.ts', included: false, mutated: true }, + 'node_modules/stryker-api/*.@(js|map)', + 'node_modules/stryker-api/src/**/*.@(js|map)', + 'package.json', + 'src/**/*.ts', '!src/**/*.d.ts', - { pattern: 'node_modules/stryker-api/*.js', included: false, mutated: false, transpiled: false }, - { pattern: 'node_modules/stryker-api/src/**/*.js', included: false, mutated: false, transpiled: false } + 'test/**/*.ts', + '!test/**/*.d.ts' ], + mutate: ['src/**/*.ts'], coverageAnalysis: 'perTest', tsconfigFile: 'tsconfig.json', mutator: 'typescript', transpilers: [ 'typescript' - ] + ], + mochaOptions: { + files: ['test/helpers/**/*.js', 'test/unit/**/*.js'] + } }) } else { config.set({ diff --git a/packages/stryker/test/helpers/producers.ts b/packages/stryker/test/helpers/producers.ts index 5ec27634ef..98a96914cd 100644 --- a/packages/stryker/test/helpers/producers.ts +++ b/packages/stryker/test/helpers/producers.ts @@ -1,11 +1,10 @@ import { TestResult, TestStatus, RunResult, RunStatus } from 'stryker-api/test_runner'; import { Mutant } from 'stryker-api/mutant'; -import { TranspileResult } from 'stryker-api/transpile'; import { Config } from 'stryker-api/config'; import * as sinon from 'sinon'; import { TestFramework, TestSelection } from 'stryker-api/test_framework'; import { MutantStatus, MatchedMutant, MutantResult, Reporter, ScoreResult } from 'stryker-api/report'; -import { MutationScoreThresholds, File, Location, TextFile, BinaryFile, FileKind, WebFile, FileDescriptor } from 'stryker-api/core'; +import { MutationScoreThresholds, File, Location } from 'stryker-api/core'; import TestableMutant from '../../src/TestableMutant'; import SourceFile from '../../src/SourceFile'; import TranspiledMutant from '../../src/TranspiledMutant'; @@ -13,14 +12,15 @@ import { Logger } from 'log4js'; import { FileCoverageData } from 'istanbul-lib-coverage'; import { CoverageMaps } from '../../src/transpiler/CoverageInstrumenterTranspiler'; import { MappedLocation } from '../../src/transpiler/SourceMapper'; +import TranspileResult from '../../src/transpiler/TranspileResult'; export type Mock = { [P in keyof T]: sinon.SinonStub; }; -export function mock(constructorFn: { new(...args: any[]): T; }): Mock; -export function mock(constructorFn: any): Mock; -export function mock(constructorFn: { new(...args: any[]): T; }): Mock { +export type Constructor = Function & { prototype: T }; + +export function mock(constructorFn: Constructor): Mock { return sinon.createStubInstance(constructorFn) as Mock; } @@ -43,6 +43,11 @@ function factory(defaults: T) { return (overrides?: Partial) => Object.assign({}, defaults, overrides); } +/** + * A 1x1 png base64 encoded + */ +export const PNG_BASE64_ENCODED = 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAAEnQAABJ0Ad5mH3gAAAAMSURBVBhXY/j//z8ABf4C/qc1gYQAAAAASUVORK5CYII='; + /** * Use this factory method to create deep test data * @param defaults @@ -66,14 +71,6 @@ export const mutantResult = factoryMethod(() => ({ range: [0, 0] })); -export const fileDescriptor = factory({ - name: 'fileName', - included: true, - mutated: true, - transpiled: true, - kind: FileKind.Text -}); - export const mutant = factoryMethod(() => ({ mutatorName: 'foobarMutator', fileName: 'file', @@ -100,15 +97,6 @@ export const logger = (): Mock => { }; }; -export const textFile = factory({ - name: 'file.js', - content: '', - mutated: true, - included: true, - transpiled: true, - kind: FileKind.Text -}); - export const mappedLocation = factoryMethod(() => ({ fileName: 'file.js', location: location() @@ -167,33 +155,6 @@ export const runResult = factoryMethod(() => ({ status: RunStatus.Complete })); -export const file = factory({ - name: 'file.js', - content: '', - mutated: true, - included: true, - transpiled: true, - kind: FileKind.Text -}); - -export const webFile = factory({ - name: 'http://example.org', - mutated: false, - included: true, - transpiled: false, - kind: FileKind.Web -}); - -export const binaryFile = factory({ - name: 'file.js', - content: Buffer.from(''), - mutated: true, - included: true, - transpiled: false, - kind: FileKind.Binary -}); - - export const mutationScoreThresholds = factory({ high: 80, low: 60, @@ -221,12 +182,16 @@ export function matchedMutant(numberOfTests: number, mutantId = numberOfTests.to }; } +export function file() { + return new File('', ''); +} + export const transpileResult = factoryMethod(() => ({ error: null, outputFiles: [file(), file()] })); -export const sourceFile = () => new SourceFile(textFile()); +export const sourceFile = () => new SourceFile(file()); export const testableMutant = (fileName = 'file', mutatorName = 'foobarMutator') => new TestableMutant('1337', mutant({ mutatorName, @@ -234,7 +199,7 @@ export const testableMutant = (fileName = 'file', mutatorName = 'foobarMutator') replacement: '-', fileName }), new SourceFile( - textFile({ name: fileName, content: 'const a = 4 + 5' }) + new File(fileName, Buffer.from('const a = 4 + 5')) )); export const transpiledMutant = (fileName = 'file') => diff --git a/packages/stryker/test/integration/child-proxy/ChildProcessProxy.it.ts b/packages/stryker/test/integration/child-proxy/ChildProcessProxy.it.ts new file mode 100644 index 0000000000..b601e181ac --- /dev/null +++ b/packages/stryker/test/integration/child-proxy/ChildProcessProxy.it.ts @@ -0,0 +1,42 @@ +import { expect } from 'chai'; +import Echo from './Echo'; +import ChildProcessProxy from '../../../src/child-proxy/ChildProcessProxy'; +import { File } from 'stryker-api/core'; + +describe('ChildProcessProxy', () => { + + let sut: ChildProcessProxy; + + beforeEach(() => { + sut = ChildProcessProxy.create(require.resolve('./Echo'), 'info', [], Echo, 'World'); + }); + + afterEach(() => { + sut.dispose(); + }); + + it('should be able to get direct result', async () => { + const actual = await sut.proxy.say('hello'); + expect(actual).eq('World: hello'); + }); + + it('should be able to get delayed result', async () => { + const actual = await sut.proxy.sayDelayed('hello', 2); + expect(actual).eq('World: hello (2 ms)'); + }); + + it('should be able to receive files', async () => { + const actual: string = await sut.proxy.echoFile(new File('hello.txt', 'hello world from file')); + expect(actual).eq('hello world from file'); + }); + + it('should be able to send files', async () => { + const actual: File = await sut.proxy.readFile(); + expect(actual.textContent).eq('hello foobar'); + expect(actual.name).eq('foobar.txt'); + }); + + it('should be able to receive a promise rejection', () => { + return expect(sut.proxy.reject('Foobar error')).rejectedWith('Foobar error'); + }); +}); diff --git a/packages/stryker/test/integration/child-proxy/ChildProcessProxySpec.ts b/packages/stryker/test/integration/child-proxy/ChildProcessProxySpec.ts deleted file mode 100644 index 0db141e79d..0000000000 --- a/packages/stryker/test/integration/child-proxy/ChildProcessProxySpec.ts +++ /dev/null @@ -1,20 +0,0 @@ -import { expect } from 'chai'; -import Echo from './Echo'; -import ChildProcessProxy from '../../../src/child-proxy/ChildProcessProxy'; - -describe('ChildProcessProxy', () => { - - it('should be able to get direct result', async () => { - const sut = ChildProcessProxy.create(require.resolve('./Echo'), 'info', [], Echo, 'World'); - const actual = await sut.proxy.say('hello'); - expect(actual).eq('World: hello'); - sut.dispose(); - }); - - it('should be able to get delayed result', async () => { - const sut = ChildProcessProxy.create(require.resolve('./Echo'), 'info', [], Echo, 'World'); - const actual = await sut.proxy.sayDelayed('hello', 2); - expect(actual).eq('World: hello (2 ms)'); - sut.dispose(); - }); -}); diff --git a/packages/stryker/test/integration/child-proxy/Echo.ts b/packages/stryker/test/integration/child-proxy/Echo.ts index 2a20ff159d..53452c85f3 100644 --- a/packages/stryker/test/integration/child-proxy/Echo.ts +++ b/packages/stryker/test/integration/child-proxy/Echo.ts @@ -1,4 +1,4 @@ - +import { File } from 'stryker-api/core'; export default class Echo { @@ -17,4 +17,15 @@ export default class Echo { }); } + echoFile(file: File) { + return file.textContent; + } + + readFile() { + return new File('foobar.txt', 'hello foobar'); + } + + reject(error: string) { + return Promise.reject(new Error(error)); + } } \ No newline at end of file diff --git a/packages/stryker/test/integration/isolated-runner/AdditionalTestRunners.ts b/packages/stryker/test/integration/isolated-runner/AdditionalTestRunners.ts index 07407a8dd4..6e84b2d593 100644 --- a/packages/stryker/test/integration/isolated-runner/AdditionalTestRunners.ts +++ b/packages/stryker/test/integration/isolated-runner/AdditionalTestRunners.ts @@ -56,6 +56,17 @@ class ErroredTestRunner extends EventEmitter implements TestRunner { } } +class RejectInitRunner implements TestRunner { + + init() { + return Promise.reject(new Error('Init was rejected')); + } + + run(options: RunOptions): Promise { + throw new Error(); + } +} + class NeverResolvedTestRunner extends EventEmitter implements TestRunner { run(options: RunOptions) { return new Promise(res => { }); @@ -67,7 +78,7 @@ class SlowInitAndDisposeTestRunner extends EventEmitter implements TestRunner { inInit: boolean; init() { - return new Promise(resolve => { + return new Promise(resolve => { this.inInit = true; setTimeout(() => { this.inInit = false; @@ -92,7 +103,7 @@ class VerifyWorkingFolderTestRunner extends EventEmitter implements TestRunner { runResult: RunResult = { status: RunStatus.Complete, tests: [] }; run(options: RunOptions) { - if (process.cwd() === __dirname) { + if (process.cwd().toLowerCase() === __dirname.toLowerCase()) { return Promise.resolve(this.runResult); } else { return Promise.reject(new Error(`Expected ${process.cwd()} to be ${__dirname}`)); @@ -108,7 +119,7 @@ class AsyncronousPromiseRejectionHandlerTestRunner extends EventEmitter implemen } run(options: RunOptions) { - this.promise.catch(() => {}); + this.promise.catch(() => { }); return Promise.resolve({ status: RunStatus.Complete, tests: [] }); } } @@ -121,4 +132,5 @@ TestRunnerFactory.instance().register('discover-regex', DiscoverRegexTestRunner) TestRunnerFactory.instance().register('direct-resolved', DirectResolvedTestRunner); TestRunnerFactory.instance().register('coverage-reporting', CoverageReportingTestRunner); TestRunnerFactory.instance().register('time-bomb', TimeBombTestRunner); -TestRunnerFactory.instance().register('async-promise-rejection-handler', AsyncronousPromiseRejectionHandlerTestRunner); \ No newline at end of file +TestRunnerFactory.instance().register('async-promise-rejection-handler', AsyncronousPromiseRejectionHandlerTestRunner); +TestRunnerFactory.instance().register('reject-init', RejectInitRunner); \ No newline at end of file diff --git a/packages/stryker/test/integration/isolated-runner/ResilientTestRunnerFactorySpec.ts b/packages/stryker/test/integration/isolated-runner/ResilientTestRunnerFactorySpec.ts index a4974cf317..c06559c7af 100644 --- a/packages/stryker/test/integration/isolated-runner/ResilientTestRunnerFactorySpec.ts +++ b/packages/stryker/test/integration/isolated-runner/ResilientTestRunnerFactorySpec.ts @@ -27,8 +27,8 @@ describe('ResilientTestRunnerFactory', function () { port: 0, 'someRegex': /someRegex/ }, - files: [], port: 0, + fileNames: [], sandboxWorkingFolder: path.resolve('./test/integration/isolated-runner') }; @@ -123,6 +123,17 @@ describe('ResilientTestRunnerFactory', function () { after(() => sut.dispose()); }); + describe('when test runner behind rejects init promise', () => { + before(() => { + sut = ResilientTestRunnerFactory.create('reject-init', options); + }); + after(() => sut.dispose()); + + it('should pass along the rejection', () => { + return expect(sut.init()).rejectedWith('Init was rejected'); + }); + }); + describe('when test runner verifies the current working folder', () => { before(() => { sut = ResilientTestRunnerFactory.create('verify-working-folder', options); @@ -143,7 +154,7 @@ describe('ResilientTestRunnerFactory', function () { it('should be able to recover from crash', () => { return sleep(101) - .then(() => sut.run({ timeout: 2000 }) + .then(() => sut.run({ timeout: 4000 }) .then(result => { expect(result.status).to.be.eq(RunStatus.Complete); expect(result.errorMessages).to.be.undefined; @@ -154,7 +165,7 @@ describe('ResilientTestRunnerFactory', function () { describe('when test runner handles promise rejections asynchronously', () => { before(() => sut = ResilientTestRunnerFactory.create('async-promise-rejection-handler', options)); - + it('should be logging the unhandled rejection errors', async () => { await sut.init(); await sut.run({ timeout: 2000 }); @@ -162,6 +173,6 @@ describe('ResilientTestRunnerFactory', function () { }); after(() => sut.dispose()); - + }); }); diff --git a/packages/stryker/test/integration/source-mapper/SourceMapperIT.ts b/packages/stryker/test/integration/source-mapper/SourceMapperIT.ts index 559a4edcf8..5fc263982d 100644 --- a/packages/stryker/test/integration/source-mapper/SourceMapperIT.ts +++ b/packages/stryker/test/integration/source-mapper/SourceMapperIT.ts @@ -1,7 +1,6 @@ import * as fs from 'mz/fs'; import * as path from 'path'; import { File } from 'stryker-api/core'; -import { textFile } from '../../helpers/producers'; import { TranspiledSourceMapper } from '../../../src/transpiler/SourceMapper'; import { expect } from 'chai'; @@ -12,15 +11,7 @@ function resolve(...filePart: string[]) { function readFiles(...files: string[]): Promise { return Promise.all(files .map(relative => resolve(relative)) - .map(fileName => fs.readFile(fileName, 'utf8').then(content => { - const isMapFile = path.extname(fileName) === '.map'; - return textFile({ - content, - name: fileName, - transpiled: !isMapFile, - mutated: !isMapFile - }); - }))); + .map(fileName => fs.readFile(fileName).then(content => new File(fileName, content)))); } describe('Source mapper integration', () => { diff --git a/packages/stryker/test/unit/InputFileResolverSpec.ts b/packages/stryker/test/unit/InputFileResolverSpec.ts deleted file mode 100644 index fde4bcb47c..0000000000 --- a/packages/stryker/test/unit/InputFileResolverSpec.ts +++ /dev/null @@ -1,267 +0,0 @@ -import { resolve } from 'path'; -import { expect } from 'chai'; -import { Logger } from 'log4js'; -import { SourceFile } from 'stryker-api/report'; -import InputFileResolver from '../../src/InputFileResolver'; -import * as sinon from 'sinon'; -import * as fileUtils from '../../src/utils/fileUtils'; -import * as path from 'path'; -import * as fs from 'mz/fs'; -import { FileDescriptor, TextFile } from 'stryker-api/core'; -import currentLogMock from '../helpers/log4jsMock'; -import BroadcastReporter from '../../src/reporters/BroadcastReporter'; -import { Mock, mock, textFile, binaryFile } from '../helpers/producers'; - -const files = (...namesWithContent: [string, string][]): TextFile[] => - namesWithContent.map((nameAndContent): TextFile => textFile({ - mutated: false, - transpiled: true, - name: path.resolve(nameAndContent[0]), - content: nameAndContent[1] - })); - -describe('InputFileResolver', () => { - let log: Mock; - let globStub: sinon.SinonStub; - let sut: InputFileResolver; - let reporter: Mock; - - beforeEach(() => { - log = currentLogMock(); - reporter = mock(BroadcastReporter); - globStub = sandbox.stub(fileUtils, 'glob'); - sandbox.stub(fs, 'readFile') - .withArgs(sinon.match.string).resolves(new Buffer(0)) // fall back - .withArgs(sinon.match.string, 'utf8').resolves('') // fall back - .withArgs(sinon.match('file1'), 'utf8').resolves('file 1 content') - .withArgs(sinon.match('file2'), 'utf8').resolves('file 2 content') - .withArgs(sinon.match('file3'), 'utf8').resolves('file 3 content') - .withArgs(sinon.match('mute1'), 'utf8').resolves('mutate 1 content') - .withArgs(sinon.match('mute2'), 'utf8').resolves('mutate 2 content'); - globStub.withArgs('mut*tion*').resolves(['/mute1.js', '/mute2.js']); - globStub.withArgs('mutation1').resolves(['/mute1.js']); - globStub.withArgs('mutation2').resolves(['/mute2.js']); - globStub.withArgs('file1').resolves(['/file1.js']); - globStub.withArgs('file2').resolves(['/file2.js']); - globStub.withArgs('file3').resolves(['/file3.js']); - globStub.withArgs('file*').resolves(['/file1.js', '/file2.js', '/file3.js']); - globStub.resolves([]); // default - }); - - describe('with mutant file expressions which result in files which are included in result of all globbing files', () => { - beforeEach(() => { - sut = new InputFileResolver(['mut*tion*'], ['file1', 'mutation1', 'file2', 'mutation2', 'file3'], reporter); - }); - - it('should result in the expected input files', async () => { - const results = await sut.resolve(); - expect(results.length).to.be.eq(5); - expect(results.map(m => m.mutated)).to.deep.equal([false, true, false, true, false]); - expect(results.map(m => m.name)).to.deep.equal([ - path.resolve('/file1.js'), - path.resolve('/mute1.js'), - path.resolve('/file2.js'), - path.resolve('/mute2.js'), - path.resolve('/file3.js')] - ); - }); - - it('should report OnAllSourceFilesRead', async () => { - await sut.resolve(); - const expected: SourceFile[] = [ - { content: 'file 1 content', path: path.resolve('/file1.js') }, - { content: 'mutate 1 content', path: path.resolve('/mute1.js') }, - { content: 'file 2 content', path: path.resolve('/file2.js') }, - { content: 'mutate 2 content', path: path.resolve('/mute2.js') }, - { content: 'file 3 content', path: path.resolve('/file3.js') } - ]; - expect(reporter.onAllSourceFilesRead).calledWith(expected); - }); - - it('should report OnSourceFileRead', async () => { - await sut.resolve(); - const expected: SourceFile[] = [ - { content: 'file 1 content', path: path.resolve('/file1.js') }, - { content: 'mutate 1 content', path: path.resolve('/mute1.js') }, - { content: 'file 2 content', path: path.resolve('/file2.js') }, - { content: 'mutate 2 content', path: path.resolve('/mute2.js') }, - { content: 'file 3 content', path: path.resolve('/file3.js') } - ]; - expected.forEach(sourceFile => expect(reporter.onSourceFileRead).calledWith(sourceFile)); - }); - }); - - describe('when supplying an InputFileDescriptor without `pattern` property', () => { - it('should result in an error', () => { - expect(() => new InputFileResolver([], [{ included: false, mutated: true }], reporter)) - .throws('File descriptor {"included":false,"mutated":true} is missing mandatory property \'pattern\'.'); - }); - }); - - describe('without mutate property, but with mutated: true in files', () => { - - beforeEach(() => { - sut = new InputFileResolver([], ['file1', { pattern: 'mutation1', included: false, mutated: true }], reporter); - }); - - it('should result in the expected input files', async () => { - const results = await sut.resolve(); - expect(results).to.deep.equal([ - textFile({ included: true, mutated: false, name: resolve('/file1.js'), content: 'file 1 content' }), - textFile({ included: false, mutated: true, name: resolve('/mute1.js'), content: 'mutate 1 content' })]); - }); - - it('should log that one file is about to be mutated', async () => { - await sut.resolve(); - expect(log.info).to.have.been.calledWith('Found 1 of 2 file(s) to be mutated.'); - }); - }); - - describe('without mutate property and without mutated: true in files', () => { - - beforeEach(() => { - sut = new InputFileResolver([], ['file1', { pattern: 'mutation1', included: false, mutated: false }], reporter); - }); - - it('should warn about dry-run', async () => { - await sut.resolve(); - expect(log.warn).to.have.been.calledWith('No files marked to be mutated, stryker will perform a dry-run without actually mutating anything.'); - }); - }); - - describe('with file expressions that resolve in different order', () => { - beforeEach(() => { - sut = new InputFileResolver([], ['fileWhichResolvesLast', 'fileWhichResolvesFirst'], reporter); - globStub.withArgs('fileWhichResolvesLast').resolves(['file1']); - globStub.withArgs('fileWhichResolvesFirst').resolves(['file2']); - }); - - it('should retain original glob order', async () => { - const results = await sut.resolve(); - expect(results.map(m => m.name.substr(m.name.length - 5))).to.deep.equal(['file1', 'file2']); - }); - }); - - describe('when selecting files to mutate which are not included', () => { - let results: FileDescriptor[]; - let error: any; - beforeEach(() => { - sut = new InputFileResolver(['mut*tion*'], ['file1'], reporter); - return sut.resolve().then(r => results = r, e => error = e); - }); - - it('should reject the result', () => { - expect(results).to.not.be.ok; - expect(error.message).to.be.eq([ - `Could not find mutate file "${path.resolve('/mute1.js')}" in list of files.`, - `Could not find mutate file "${path.resolve('/mute2.js')}" in list of files.` - ].join(' ')); - }); - }); - - describe('when a globbing expression does not result in a result', () => { - beforeEach(() => { - sut = new InputFileResolver(['file1'], ['file1', 'notExists'], reporter); - }); - - it('should log a warning', async () => { - await sut.resolve(); - expect(log.warn).to.have.been.calledWith('Globbing expression "notExists" did not result in any files.'); - }); - }); - - describe('when a globbing expression results in a reject', () => { - let results: FileDescriptor[]; - let actualError: any; - let expectedError: Error; - - beforeEach(() => { - sut = new InputFileResolver(['file1'], ['fileError', 'fileError'], reporter); - expectedError = new Error('ERROR: something went wrong'); - globStub.withArgs('fileError').rejects(expectedError); - return sut.resolve().then(r => results = r, e => actualError = e); - }); - - it('should reject the promise', () => { - expect(results).to.not.be.ok; - expect(actualError).to.deep.equal(expectedError); - }); - }); - - describe('when excluding files with "!"', () => { - - it('should exclude the files that were previously included', () => { - return expect(new InputFileResolver([], ['file2', 'file1', '!file2'], reporter).resolve()) - .to.eventually.deep.equal(files(['/file1.js', 'file 1 content'])); - }); - - it('should exclude the files that were previously with a wild card', () => { - return expect(new InputFileResolver([], ['file*', '!file2'], reporter).resolve()) - .to.eventually.deep.equal(files(['/file1.js', 'file 1 content'], ['/file3.js', 'file 3 content'])); - }); - - it('should not exclude files added using an input file descriptor', () => { - return expect(new InputFileResolver([], ['file2', { pattern: '!file2' }], reporter).resolve()) - .to.eventually.deep.equal(files(['/file2.js', 'file 2 content'])); - }); - - it('should not exclude files when the globbing expression results in an empty array', () => { - return expect(new InputFileResolver([], ['file2', '!does/not/exist'], reporter).resolve()) - .to.eventually.deep.equal(files(['/file2.js', 'file 2 content'])); - }); - }); - - describe('when provided duplicate files', () => { - - it('should deduplicate files that occur more than once', () => { - return expect(new InputFileResolver([], ['file2', 'file2'], reporter).resolve()) - .to.eventually.deep.equal(files(['/file2.js', 'file 2 content'])); - }); - - it('should deduplicate files that previously occurred in a wildcard expression', () => { - return expect(new InputFileResolver([], ['file*', 'file2'], reporter).resolve()) - .to.eventually.deep.equal(files(['/file1.js', 'file 1 content'], ['/file2.js', 'file 2 content'], ['/file3.js', 'file 3 content'])); - }); - - it('should order files by expression order', () => { - return expect(new InputFileResolver([], ['file2', 'file*'], reporter).resolve()) - .eventually.deep.equal(files(['/file2.js', 'file 2 content'], ['/file1.js', 'file 1 content'], ['/file3.js', 'file 3 content'])); - }); - }); - - describe('with url as file pattern', () => { - it('should pass through the web urls without globbing', () => { - return new InputFileResolver([], ['http://www', { pattern: 'https://ok' }], reporter) - .resolve() - .then(() => expect(fileUtils.glob).to.not.have.been.called); - }); - - it('should fail when web url is in the mutated array', () => { - expect(() => new InputFileResolver(['http://www'], ['http://www'], reporter)) - .throws('Cannot mutate web url "http://www".'); - }); - - it('should fail when web url is to be mutated', () => { - expect(() => new InputFileResolver([], [{ pattern: 'http://www', mutated: true }], reporter)) - .throws('Cannot mutate web url "http://www".'); - }); - }); - - - it(`should presume that files with an extension like .jpeg or .gif are binary`, async () => { - // Arrange - const binaryFiles = ['file.png', 'file.jpeg', 'file.jpg', 'file.gif', 'file.zip', 'file.tar']; - binaryFiles.forEach(file => globStub.withArgs(file).resolves([file])); - sut = new InputFileResolver([], binaryFiles, reporter); - - // Act - const actual = await sut.resolve(); - - // Assert - expect(actual).deep.eq(binaryFiles.map(file => binaryFile({ - name: path.resolve(file), - mutated: false, - transpiled: true - }))); - }); -}); \ No newline at end of file diff --git a/packages/stryker/test/unit/MutantTestMatcherSpec.ts b/packages/stryker/test/unit/MutantTestMatcherSpec.ts index a0be4b9c6a..e1c01f7fe5 100644 --- a/packages/stryker/test/unit/MutantTestMatcherSpec.ts +++ b/packages/stryker/test/unit/MutantTestMatcherSpec.ts @@ -1,14 +1,13 @@ import { Logger } from 'log4js'; import { Mutant } from 'stryker-api/mutant'; import { TestSelection } from 'stryker-api/test_framework'; -import { textFile, testResult } from './../helpers/producers'; import { expect } from 'chai'; import { RunResult, TestResult, RunStatus, TestStatus, CoverageCollection, CoveragePerTestResult } from 'stryker-api/test_runner'; import { StrykerOptions, File } from 'stryker-api/core'; import { MatchedMutant } from 'stryker-api/report'; import MutantTestMatcher from '../../src/MutantTestMatcher'; import currentLogMock from '../helpers/log4jsMock'; -import { file, mutant, Mock, mock } from '../helpers/producers'; +import { testResult, mutant, Mock, mock } from '../helpers/producers'; import TestableMutant, { TestSelectionResult } from '../../src/TestableMutant'; import SourceFile from '../../src/SourceFile'; import BroadcastReporter from '../../src/reporters/BroadcastReporter'; @@ -24,7 +23,7 @@ describe('MutantTestMatcher', () => { let fileCoverageDictionary: CoverageMapsByFile; let strykerOptions: StrykerOptions; let reporter: Mock; - let files: File[]; + let filesToMutate: ReadonlyArray; let sourceMapper: PassThroughSourceMapper; beforeEach(() => { @@ -34,18 +33,12 @@ describe('MutantTestMatcher', () => { runResult = { tests: [], status: RunStatus.Complete }; strykerOptions = {}; reporter = mock(BroadcastReporter); - files = [file({ - name: 'fileWithMutantOne', - content: '\n\n\n\n12345' - }), file({ - name: 'fileWithMutantTwo', - content: '\n\n\n\n\n\n\n\n\n\n' - })]; + filesToMutate = [new File('fileWithMutantOne', '\n\n\n\n12345'), new File('fileWithMutantTwo', '\n\n\n\n\n\n\n\n\n\n')]; sourceMapper = new PassThroughSourceMapper(); sandbox.spy(sourceMapper, 'transpiledLocationFor'); sut = new MutantTestMatcher( mutants, - files, + filesToMutate, runResult, sourceMapper, fileCoverageDictionary, @@ -102,8 +95,8 @@ describe('MutantTestMatcher', () => { const expectedTestSelection = [{ id: 0, name: 'test one' }, { id: 1, name: 'test two' }]; expect(result[0].selectedTests).deep.eq(expectedTestSelection); expect(result[1].selectedTests).deep.eq(expectedTestSelection); - expect(result[0].testSelectionResult).eq(TestSelectionResult.FailedButAlreadyReporter); - expect(result[1].testSelectionResult).eq(TestSelectionResult.FailedButAlreadyReporter); + expect(result[0].testSelectionResult).eq(TestSelectionResult.FailedButAlreadyReported); + expect(result[1].testSelectionResult).eq(TestSelectionResult.FailedButAlreadyReported); expect(log.warn).calledWith('No coverage result found, even though coverageAnalysis is "%s". Assuming that all tests cover each mutant. This might have a big impact on the performance.', 'perTest'); }); @@ -317,7 +310,7 @@ describe('MutantTestMatcher', () => { describe('should not result in regression', () => { it('should match up mutant for issue #151 (https://github.com/stryker-mutator/stryker/issues/151)', () => { - const sourceFile = new SourceFile(textFile()); + const sourceFile = new SourceFile(new File('', '')); sourceFile.getLocation = () => ({ 'start': { 'line': 13, 'column': 38 }, 'end': { 'line': 24, 'column': 5 } }); const testableMutant = new TestableMutant('1', mutant({ fileName: 'juice-shop\\app\\js\\controllers\\SearchResultController.js' @@ -353,8 +346,8 @@ describe('MutantTestMatcher', () => { const expectedTestSelection: TestSelection[] = [{ id: 0, name: 'name' }, { id: 1, name: 'name' }]; expect(result[0].selectedTests).deep.eq(expectedTestSelection); expect(result[1].selectedTests).deep.eq(expectedTestSelection); - expect(result[0].testSelectionResult).deep.eq(TestSelectionResult.FailedButAlreadyReporter); - expect(result[1].testSelectionResult).deep.eq(TestSelectionResult.FailedButAlreadyReporter); + expect(result[0].testSelectionResult).deep.eq(TestSelectionResult.FailedButAlreadyReported); + expect(result[1].testSelectionResult).deep.eq(TestSelectionResult.FailedButAlreadyReported); expect(log.warn).to.have.been.calledWith('No coverage result found, even though coverageAnalysis is "%s". Assuming that all tests cover each mutant. This might have a big impact on the performance.', 'all'); }); diff --git a/packages/stryker/test/unit/SandboxPoolSpec.ts b/packages/stryker/test/unit/SandboxPoolSpec.ts index 50345e7aa8..ac48ac75b1 100644 --- a/packages/stryker/test/unit/SandboxPoolSpec.ts +++ b/packages/stryker/test/unit/SandboxPoolSpec.ts @@ -4,7 +4,7 @@ import { expect } from 'chai'; import { Config } from 'stryker-api/config'; import SandboxPool from '../../src/SandboxPool'; import { TestFramework } from 'stryker-api/test_framework'; -import { Mock, mock, testFramework, textFile, config } from '../helpers/producers'; +import { Mock, mock, testFramework, file, config } from '../helpers/producers'; import Sandbox from '../../src/Sandbox'; import '../helpers/globals'; @@ -30,7 +30,7 @@ describe('SandboxPool', () => { .onCall(0).resolves(firstSandbox) .onCall(1).resolves(secondSandbox); - expectedInputFiles = [textFile()]; + expectedInputFiles = [file()]; sut = new SandboxPool(options, expectedTestFramework, expectedInputFiles); }); diff --git a/packages/stryker/test/unit/SandboxSpec.ts b/packages/stryker/test/unit/SandboxSpec.ts index b29d751aa4..faf28e53e9 100644 --- a/packages/stryker/test/unit/SandboxSpec.ts +++ b/packages/stryker/test/unit/SandboxSpec.ts @@ -5,31 +5,29 @@ import * as sinon from 'sinon'; import * as path from 'path'; import * as mkdirp from 'mkdirp'; import { expect } from 'chai'; -import { FileKind, File } from 'stryker-api/core'; -import { TextFile } from 'stryker-api/src/core/File'; +import { File } from 'stryker-api/core'; import { wrapInClosure } from '../../src/utils/objectUtils'; import Sandbox from '../../src/Sandbox'; import { TempFolder } from '../../src/utils/TempFolder'; import ResilientTestRunnerFactory from '../../src/isolated-runner/ResilientTestRunnerFactory'; import IsolatedRunnerOptions from '../../src/isolated-runner/IsolatedRunnerOptions'; import TestableMutant, { TestSelectionResult } from '../../src/TestableMutant'; -import { mutant as createMutant, testResult, textFile, fileDescriptor, webFile, transpileResult, Mock } from '../helpers/producers'; +import { mutant as createMutant, testResult, Mock } from '../helpers/producers'; import SourceFile from '../../src/SourceFile'; import '../helpers/globals'; import TranspiledMutant from '../../src/TranspiledMutant'; import * as fileUtils from '../../src/utils/fileUtils'; import currentLogMock from '../helpers/log4jsMock'; +import TestRunnerDecorator from '../../src/isolated-runner/TestRunnerDecorator'; describe('Sandbox', () => { let sut: Sandbox; let options: Config; - let textFiles: TextFile[]; let files: File[]; - let testRunner: any; + let testRunner: Mock; let testFrameworkStub: any; - let expectedFileToMutate: TextFile; - let notMutatedFile: TextFile; - let webFileUrl: string; + let expectedFileToMutate: File; + let notMutatedFile: File; let workingFolder: string; let expectedTargetFileToMutate: string; let expectedTestFrameworkHooksFile: string; @@ -38,21 +36,19 @@ describe('Sandbox', () => { beforeEach(() => { options = { port: 43, timeoutFactor: 23, timeoutMs: 1000, testRunner: 'sandboxUnitTestRunner' } as any; - testRunner = { init: sandbox.stub(), run: sandbox.stub().resolves() }; + testRunner = { init: sandbox.stub(), run: sandbox.stub().resolves(), dispose: sandbox.stub() }; testFrameworkStub = { filter: sandbox.stub() }; - expectedFileToMutate = textFile({ name: path.resolve('file1'), content: 'original code', mutated: true, included: true }); - notMutatedFile = textFile({ name: path.resolve('file2'), content: 'to be mutated', mutated: false, included: false }); - webFileUrl = 'https://ajax.googleapis.com/ajax/libs/angularjs/1.3.12/angular.js'; - workingFolder = 'random-folder-3'; + expectedFileToMutate = new File(path.resolve('file1'), 'original code'); + notMutatedFile = new File(path.resolve('file2'), 'to be mutated'); + workingFolder = path.resolve('random-folder-3'); expectedTargetFileToMutate = path.join(workingFolder, 'file1'); expectedTestFrameworkHooksFile = path.join(workingFolder, '___testHooksForStryker.js'); - textFiles = [ + files = [ expectedFileToMutate, notMutatedFile, ]; - files = (textFiles as File[]).concat([webFile({ name: webFileUrl, mutated: false, included: true, transpiled: false })]); sandbox.stub(TempFolder.instance(), 'createRandomFolder').returns(workingFolder); fileSystemStub = sandbox.stub(fileUtils, 'writeFile'); fileSystemStub.resolves(); @@ -63,13 +59,13 @@ describe('Sandbox', () => { it('should copy input files when created', async () => { sut = await Sandbox.create(options, 3, files, null); - expect(fileUtils.writeFile).calledWith(expectedTargetFileToMutate, textFiles[0].content); - expect(fileUtils.writeFile).calledWith(path.join(workingFolder, 'file2'), textFiles[1].content); + expect(fileUtils.writeFile).calledWith(expectedTargetFileToMutate, files[0].content); + expect(fileUtils.writeFile).calledWith(path.join(workingFolder, 'file2'), files[1].content); }); it('should copy a local file when created', async () => { - sut = await Sandbox.create(options, 3, [textFile({ name: 'localFile.js', content: 'foobar' })], null); - expect(fileUtils.writeFile).calledWith(path.join(workingFolder, 'localFile.js'), 'foobar'); + sut = await Sandbox.create(options, 3, [new File('localFile.js', 'foobar')], null); + expect(fileUtils.writeFile).calledWith(path.join(workingFolder, 'localFile.js'), Buffer.from('foobar')); }); describe('when constructed with a testFramework', () => { @@ -78,34 +74,28 @@ describe('Sandbox', () => { sut = await Sandbox.create(options, 3, files, testFrameworkStub); }); - it('should not have written online files', () => { - let expectedBaseFolder = webFileUrl.substr(workingFolder.length - 1); // The Sandbox expects all files to be absolute paths. An online file is not an absolute path. - - expect(mkdirp.sync).not.calledWith(workingFolder + path.dirname(expectedBaseFolder)); - expect(fileUtils.writeFile).not.calledWith(webFileUrl, sinon.match.any, sinon.match.any); - }); - it('should have created a workingFolder', () => { expect(TempFolder.instance().createRandomFolder).to.have.been.calledWith('sandbox'); }); it('should have created the isolated test runner without framework hook', () => { const expectedSettings: IsolatedRunnerOptions = { - files: [ - fileDescriptor({ name: expectedTestFrameworkHooksFile, mutated: false, included: true, transpiled: false }), - fileDescriptor({ name: expectedTargetFileToMutate, mutated: true, included: true }), - fileDescriptor({ name: path.join(workingFolder, 'file2'), mutated: false, included: false }), - fileDescriptor({ name: webFileUrl, mutated: false, included: true, kind: FileKind.Web, transpiled: false }) - ], port: 46, strykerOptions: options, - sandboxWorkingFolder: workingFolder + sandboxWorkingFolder: workingFolder, + fileNames: [path.resolve('random-folder-3', 'file1'), path.resolve('random-folder-3', 'file2')] }; expect(ResilientTestRunnerFactory.create).calledWith(options.testRunner, expectedSettings); }); describe('when run', () => { - it('should run the testRunner', () => sut.run(231313).then(() => expect(testRunner.run).to.have.been.calledWith({ timeout: 231313 }))); + it('should run the testRunner', async () => { + await sut.run(231313, 'hooks'); + expect(testRunner.run).to.have.been.calledWith({ + timeout: 231313, + testHooks: 'hooks' + }); + }); }); describe('when runMutant()', () => { @@ -119,24 +109,21 @@ describe('Sandbox', () => { const testableMutant = new TestableMutant( '1', mutant, - new SourceFile(textFile({ content: 'original code' }))); + new SourceFile(new File('foobar.js', 'original code'))); testableMutant.selectTest(testResult({ timeSpentMs: 10 }), 1); testableMutant.selectTest(testResult({ timeSpentMs: 2 }), 2); - transpiledMutant = new TranspiledMutant(testableMutant, { - error: null, - outputFiles: [textFile({ name: expectedFileToMutate.name, content: 'mutated code' })] - }, true); + transpiledMutant = new TranspiledMutant(testableMutant, { outputFiles: [new File(expectedFileToMutate.name, 'mutated code')], error: null }, true); testFrameworkStub.filter.returns(testFilterCodeFragment); }); it('should save the mutant to disk', async () => { await sut.runMutant(transpiledMutant); - expect(fileUtils.writeFile).to.have.been.calledWith(expectedTargetFileToMutate, 'mutated code'); + expect(fileUtils.writeFile).calledWith(expectedTargetFileToMutate, Buffer.from('mutated code')); expect(log.warn).not.called; }); it('should nog log a warning if test selection was failed but already reported', async () => { - transpiledMutant.mutant.testSelectionResult = TestSelectionResult.FailedButAlreadyReporter; + transpiledMutant.mutant.testSelectionResult = TestSelectionResult.FailedButAlreadyReported; await sut.runMutant(transpiledMutant); expect(log.warn).not.called; }); @@ -153,14 +140,10 @@ describe('Sandbox', () => { expect(testFrameworkStub.filter).to.have.been.calledWith(transpiledMutant.mutant.selectedTests); }); - it('should write the filter code fragment to hooks file', async () => { - await sut.runMutant(transpiledMutant); - expect(fileUtils.writeFile).calledWith(expectedTestFrameworkHooksFile, wrapInClosure(testFilterCodeFragment)); - }); - - it('should have ran testRunner with correct timeout', async () => { + it('should provide the filter code as testHooks and correct timeout', async () => { await sut.runMutant(transpiledMutant); - expect(testRunner.run).calledWith({ timeout: 12 * 23 + 1000 }); + const expectedRunOptions = { testHooks: wrapInClosure(testFilterCodeFragment), timeout: 12 * 23 + 1000 }; + expect(testRunner.run).calledWith(expectedRunOptions); }); it('should have reset the source file', async () => { @@ -168,7 +151,7 @@ describe('Sandbox', () => { let timesCalled = fileSystemStub.getCalls().length - 1; let lastCall = fileSystemStub.getCall(timesCalled); - expect(lastCall.args).to.deep.equal([expectedTargetFileToMutate, 'original code']); + expect(lastCall.args).to.deep.equal([expectedTargetFileToMutate, Buffer.from('original code')]); }); }); }); @@ -181,14 +164,10 @@ describe('Sandbox', () => { it('should have created the isolated test runner', () => { const expectedSettings: IsolatedRunnerOptions = { - files: [ - fileDescriptor({ name: path.join(workingFolder, 'file1'), mutated: true, included: true }), - fileDescriptor({ name: path.join(workingFolder, 'file2'), mutated: false, included: false }), - fileDescriptor({ name: webFileUrl, mutated: false, included: true, transpiled: false, kind: FileKind.Web }) - ], port: 46, strykerOptions: options, - sandboxWorkingFolder: workingFolder + sandboxWorkingFolder: workingFolder, + fileNames: [path.resolve('random-folder-3', 'file1'), path.resolve('random-folder-3', 'file2')] }; expect(ResilientTestRunnerFactory.create).to.have.been.calledWith(options.testRunner, expectedSettings); }); @@ -196,8 +175,8 @@ describe('Sandbox', () => { describe('when runMutant()', () => { beforeEach(() => { - const mutant = new TestableMutant('2', createMutant(), new SourceFile(textFile())); - return sut.runMutant(new TranspiledMutant(mutant, transpileResult({ outputFiles: [textFile({ name: expectedTargetFileToMutate })] }), true)); + const mutant = new TestableMutant('2', createMutant(), new SourceFile(new File('', ''))); + return sut.runMutant(new TranspiledMutant(mutant, { outputFiles: [new File(expectedTargetFileToMutate, '')], error: null }, true)); }); it('should not filter any tests', () => { diff --git a/packages/stryker/test/unit/SourceFileSpec.ts b/packages/stryker/test/unit/SourceFileSpec.ts index fb9a76bdcc..274ac94d55 100644 --- a/packages/stryker/test/unit/SourceFileSpec.ts +++ b/packages/stryker/test/unit/SourceFileSpec.ts @@ -1,6 +1,6 @@ import { expect } from 'chai'; +import { File } from 'stryker-api/core'; import SourceFile from '../../src/SourceFile'; -import { textFile } from '../helpers/producers'; const content = ` @@ -17,7 +17,7 @@ describe('SourceFile', () => { let sut: SourceFile; beforeEach(() => { - sut = new SourceFile(textFile({ content })); + sut = new SourceFile(new File('', content)); }); describe('getLocation', () => { @@ -40,7 +40,7 @@ describe('SourceFile', () => { }); it('should work for line 0', () => { - sut = new SourceFile(textFile({ content: '1234567' })); + sut = new SourceFile(new File('', '1234567')); expect(sut.getLocation([2, 4])).deep.eq({ start: { line: 0, column: 2 }, end: { line: 0, column: 4 }}); }); }); diff --git a/packages/stryker/test/unit/StrykerSpec.ts b/packages/stryker/test/unit/StrykerSpec.ts index 3f57a7d2ae..598a88fd1b 100644 --- a/packages/stryker/test/unit/StrykerSpec.ts +++ b/packages/stryker/test/unit/StrykerSpec.ts @@ -5,7 +5,7 @@ import { Config, ConfigEditorFactory, ConfigEditor } from 'stryker-api/config'; import { RunResult } from 'stryker-api/test_runner'; import { TestFramework } from 'stryker-api/test_framework'; import { expect } from 'chai'; -import InputFileResolver, * as inputFileResolver from '../../src/InputFileResolver'; +import InputFileResolver, * as inputFileResolver from '../../src/input/InputFileResolver'; import ConfigReader, * as configReader from '../../src/ConfigReader'; import TestFrameworkOrchestrator, * as testFrameworkOrchestrator from '../../src/TestFrameworkOrchestrator'; import ReporterOrchestrator, * as reporterOrchestrator from '../../src/ReporterOrchestrator'; @@ -18,10 +18,11 @@ import ScoreResultCalculator, * as scoreResultCalculatorModule from '../../src/S import PluginLoader, * as pluginLoader from '../../src/PluginLoader'; import { TempFolder } from '../../src/utils/TempFolder'; import currentLogMock from '../helpers/log4jsMock'; -import { mock, Mock, testFramework as testFrameworkMock, textFile, config, runResult, testableMutant, mutantResult } from '../helpers/producers'; +import { mock, Mock, testFramework as testFrameworkMock, config, runResult, testableMutant, mutantResult } from '../helpers/producers'; import BroadcastReporter from '../../src/reporters/BroadcastReporter'; import TestableMutant from '../../src/TestableMutant'; import '../helpers/globals'; +import InputFileCollection from '../../src/input/InputFileCollection'; class FakeConfigEditor implements ConfigEditor { constructor() { } @@ -117,7 +118,7 @@ describe('Stryker', function () { describe('runMutationTest()', () => { - let inputFiles: File[]; + let inputFiles: InputFileCollection; let initialRunResult: RunResult; let transpiledFiles: File[]; let mutants: TestableMutant[]; @@ -133,8 +134,8 @@ describe('Stryker', function () { mutantRunResultMatcherMock.matchWithMutants.returns(mutants); mutatorMock.mutate.returns(mutants); mutationTestExecutorMock.run.resolves(mutantResults); - inputFiles = [textFile({ name: 'input.ts ' })]; - transpiledFiles = [textFile({ name: 'output.js' })]; + inputFiles = new InputFileCollection([new File('input.ts', '')], ['input.ts']); + transpiledFiles = [new File('output.js', '')]; inputFileResolverMock.resolve.resolves(inputFiles); initialRunResult = runResult(); initialTestExecutorMock.run.resolves({ runResult: initialRunResult, transpiledFiles }); @@ -217,15 +218,15 @@ describe('Stryker', function () { expect(initialTestExecutorMock.run).called; }); - it('should create the mutant generator', () => { + it('should create the mutator', () => { expect(mutatorFacade.default).calledWithNew; expect(mutatorFacade.default).calledWith(strykerConfig); - expect(mutatorMock.mutate).calledWith(inputFiles); + expect(mutatorMock.mutate).calledWith(inputFiles.filesToMutate); }); it('should create the mutation test executor', () => { expect(mutationTestExecutor.default).calledWithNew; - expect(mutationTestExecutor.default).calledWith(strykerConfig, inputFiles, testFramework, reporter); + expect(mutationTestExecutor.default).calledWith(strykerConfig, inputFiles.files, testFramework, reporter); expect(mutationTestExecutorMock.run).calledWith(mutants); }); diff --git a/packages/stryker/test/unit/TestableMutantSpec.ts b/packages/stryker/test/unit/TestableMutantSpec.ts index 60df5498a8..42b44fee18 100644 --- a/packages/stryker/test/unit/TestableMutantSpec.ts +++ b/packages/stryker/test/unit/TestableMutantSpec.ts @@ -1,53 +1,53 @@ import { expect } from 'chai'; import { Mutant } from 'stryker-api/mutant'; import TestableMutant, { TestSelectionResult } from '../../src/TestableMutant'; -import { mutant, textFile, runResult, testResult } from './../helpers/producers'; +import { mutant, runResult, testResult } from './../helpers/producers'; import SourceFile from '../../src/SourceFile'; -import { TextFile } from 'stryker-api/core'; +import { File } from 'stryker-api/core'; describe('TestableMutant', () => { - let sut: TestableMutant; let innerMutant: Mutant; - let innerTextFile: TextFile; beforeEach(() => { innerMutant = mutant(); - innerTextFile = textFile(); - sut = new TestableMutant('3', innerMutant, new SourceFile(innerTextFile)); }); it('should pass properties from mutant and source code', () => { + const sut = new TestableMutant('3', innerMutant, new SourceFile(new File(innerMutant.fileName, 'alert("foobar")'))); expect(sut.id).eq('3'); expect(sut.fileName).eq(innerMutant.fileName); expect(sut.range).eq(innerMutant.range); expect(sut.mutatorName).eq(innerMutant.mutatorName); expect(sut.replacement).eq(innerMutant.replacement); - expect(sut.originalCode).eq(innerTextFile.content); + expect(sut.originalCode).eq('alert("foobar")'); }); it('should reflect timeSpentScopedTests, scopedTestIds and TestSelectionResult', () => { - sut.selectAllTests(runResult({ tests: [testResult({ name: 'spec1', timeSpentMs: 12 }), testResult({ name: 'spec2', timeSpentMs: 42 })] }), TestSelectionResult.FailedButAlreadyReporter); + const sut = new TestableMutant('3', innerMutant, new SourceFile(new File('foobar.js', 'alert("foobar")'))); + sut.selectAllTests(runResult({ tests: [testResult({ name: 'spec1', timeSpentMs: 12 }), testResult({ name: 'spec2', timeSpentMs: 42 })] }), TestSelectionResult.FailedButAlreadyReported); expect(sut.timeSpentScopedTests).eq(54); expect(sut.selectedTests).deep.eq([{ id: 0, name: 'spec1' }, { id: 1, name: 'spec2' }]); - expect(sut.testSelectionResult).eq(TestSelectionResult.FailedButAlreadyReporter); + expect(sut.testSelectionResult).eq(TestSelectionResult.FailedButAlreadyReported); }); it('should calculate position using sourceFile', () => { - innerTextFile.content = 'some content'; + const sut = new TestableMutant('3', innerMutant, new SourceFile(new File('foobar.js', 'some content'))); innerMutant.range = [1, 2]; expect(sut.location).deep.eq({ start: { line: 0, column: 1 }, end: { line: 0, column: 2 } }); }); - + it('should return original code with mutant replacement when `mutatedCode` is requested', () => { - innerTextFile.content = 'some content'; + const innerFile = new File('', 'some content'); + const sut = new TestableMutant('3', innerMutant, new SourceFile(innerFile)); innerMutant.range = [4, 5]; innerMutant.replacement = ' mutated! '; expect(sut.mutatedCode).eq('some mutated! content'); }); - + it('should be able to retrieve original lines and mutated lines', () => { - innerTextFile.content = 'line 1\nline 2\nline 3\nline 4'; + const innerFile = new File('', 'line 1\nline 2\nline 3\nline 4'); + const sut = new TestableMutant('3', innerMutant, new SourceFile(innerFile)); innerMutant.range = [11, 12]; innerMutant.replacement = ' mutated! '; expect(sut.originalLines).eq('line 2'); diff --git a/packages/stryker/test/unit/child-proxy/ChildProcessProxySpec.ts b/packages/stryker/test/unit/child-proxy/ChildProcessProxySpec.ts index 5ee6349f6f..9a604a6bee 100644 --- a/packages/stryker/test/unit/child-proxy/ChildProcessProxySpec.ts +++ b/packages/stryker/test/unit/child-proxy/ChildProcessProxySpec.ts @@ -1,7 +1,7 @@ import { expect } from 'chai'; import * as childProcess from 'child_process'; import ChildProcessProxy from '../../../src/child-proxy/ChildProcessProxy'; -import { autoStart, InitMessage, WorkerMessageKind, ParentMessage, WorkerMessage } from '../../../src/child-proxy/messageProtocol'; +import { autoStart, InitMessage, WorkerMessageKind, ParentMessage, WorkerMessage, ParentMessageKind } from '../../../src/child-proxy/messageProtocol'; import { serialize } from '../../../src/utils/objectUtils'; import HelloClass from './HelloClass'; @@ -53,11 +53,11 @@ describe('ChildProcessProxy', () => { }); }); - describe('when calling messages', () => { + describe('when calling methods', () => { beforeEach(() => { sut = ChildProcessProxy.create('', '', [], HelloClass, ''); - const initDoneResult: ParentMessage = 'init_done'; + const initDoneResult: ParentMessage = { kind: ParentMessageKind.Initialized }; const msg = serialize(initDoneResult); childProcessMock.on.callArgWith(1, [msg]); }); @@ -65,6 +65,7 @@ describe('ChildProcessProxy', () => { it('should proxy the message', async () => { // Arrange const workerResponse: ParentMessage = { + kind: ParentMessageKind.Result, correlationId: 0, result: 'ack' }; diff --git a/packages/stryker/test/unit/child-proxy/ChildProcessWorkerSpec.ts b/packages/stryker/test/unit/child-proxy/ChildProcessWorkerSpec.ts index e64fd7a967..435a4303f1 100644 --- a/packages/stryker/test/unit/child-proxy/ChildProcessWorkerSpec.ts +++ b/packages/stryker/test/unit/child-proxy/ChildProcessWorkerSpec.ts @@ -1,7 +1,7 @@ import ChildProcessProxyWorker from '../../../src/child-proxy/ChildProcessProxyWorker'; import { expect } from 'chai'; import { serialize } from '../../../src/utils/objectUtils'; -import { WorkerMessage, WorkerMessageKind, ParentMessage, WorkResult, WorkMessage } from '../../../src/child-proxy/messageProtocol'; +import { WorkerMessage, WorkerMessageKind, ParentMessage, WorkResult, WorkMessage, ParentMessageKind } from '../../../src/child-proxy/messageProtocol'; import * as log4js from 'log4js'; import PluginLoader, * as pluginLoader from '../../../src/PluginLoader'; import { Mock, mock } from '../../helpers/producers'; @@ -57,21 +57,21 @@ describe('ChildProcessProxyWorker', () => { requirePath: require.resolve('./HelloClass') }; }); - + it('should create the correct real instance', () => { processOnStub.callArgWith(1, [serialize(initMessage)]); expect(sut.realSubject).instanceOf(HelloClass); const actual = sut.realSubject as HelloClass; expect(actual.name).eq('FooBarName'); }); - + it('should send "init_done"', async () => { processOnStub.callArgWith(1, [serialize(initMessage)]); - const expectedWorkerResponse: ParentMessage = 'init_done'; + const expectedWorkerResponse: ParentMessage = { kind: ParentMessageKind.Initialized }; await tick(); // make sure promise is resolved expect(processSendStub).calledWith(serialize(expectedWorkerResponse)); }); - + it('should remove any additional listeners', async () => { // Arrange function noop() { } @@ -84,14 +84,14 @@ describe('ChildProcessProxyWorker', () => { // Assert expect(processRemoveListenerStub).calledWith('message', noop); }); - + it('should set global log level', () => { - processOnStub.callArgWith(1, [serialize(initMessage)]); + processOnStub.callArgWith(1, serialize(initMessage)); expect(setGlobalLogLevelStub).calledWith('FooLevel'); }); - + it('should load plugins', () => { - processOnStub.callArgWith(1, [serialize(initMessage)]); + processOnStub.callArgWith(1, serialize(initMessage)); expect(pluginLoader.default).calledWithNew; expect(pluginLoader.default).calledWith(['fooPlugin', 'barPlugin']); expect(pluginLoaderMock.load).called; @@ -101,13 +101,23 @@ describe('ChildProcessProxyWorker', () => { async function actAndAssert(workerMessage: WorkMessage, expectedResult: WorkResult) { // Act - processOnStub.callArgWith(1, [serialize(initMessage)]); + processOnStub.callArgWith(1, serialize(initMessage)); processOnStub.callArgWith(1, serialize(workerMessage)); await tick(); // Assert expect(processSendStub).calledWith(serialize(expectedResult)); } + async function actAndAssertRejection(workerMessage: WorkMessage, expectedError: string) { + // Act + processOnStub.callArgWith(1, serialize(initMessage)); + processOnStub.callArgWith(1, serialize(workerMessage)); + await tick(); + // Assert + expect(processSendStub).calledWithMatch(`"correlationId": ${workerMessage.correlationId.toString()}`); + expect(processSendStub).calledWithMatch(`"kind": ${ParentMessageKind.Rejection.toString()}`); + expect(processSendStub).calledWithMatch(`"error": "Error: ${expectedError}`); + } it('should send the result', async () => { // Arrange @@ -118,6 +128,7 @@ describe('ChildProcessProxyWorker', () => { methodName: 'sayHello' }; const expectedResult: WorkResult = { + kind: ParentMessageKind.Result, correlationId: 32, result: 'hello from FooBarName' }; @@ -125,6 +136,28 @@ describe('ChildProcessProxyWorker', () => { await actAndAssert(workerMessage, expectedResult); }); + it('should send a rejection', async () => { + // Arrange + const workerMessage: WorkerMessage = { + kind: WorkerMessageKind.Work, + correlationId: 32, + args: [], + methodName: 'reject' + }; + await actAndAssertRejection(workerMessage, 'Rejected'); + }); + + it('should send a thrown synchronous error as rejection', async () => { + // Arrange + const workerMessage: WorkerMessage = { + kind: WorkerMessageKind.Work, + correlationId: 32, + args: ['foo bar'], + methodName: 'throw' + }; + await actAndAssertRejection(workerMessage, 'foo bar'); + }); + it('should use correct arguments', async () => { // Arrange const workerMessage: WorkerMessage = { @@ -134,6 +167,7 @@ describe('ChildProcessProxyWorker', () => { methodName: 'say' }; const expectedResult: WorkResult = { + kind: ParentMessageKind.Result, correlationId: 32, result: 'hello foo and bar and chair' }; @@ -150,6 +184,7 @@ describe('ChildProcessProxyWorker', () => { methodName: 'sayDelayed' }; const expectedResult: WorkResult = { + kind: ParentMessageKind.Result, correlationId: 32, result: 'delayed hello from FooBarName' }; diff --git a/packages/stryker/test/unit/child-proxy/HelloClass.ts b/packages/stryker/test/unit/child-proxy/HelloClass.ts index 1bb304bf84..05e895ecdb 100644 --- a/packages/stryker/test/unit/child-proxy/HelloClass.ts +++ b/packages/stryker/test/unit/child-proxy/HelloClass.ts @@ -10,4 +10,12 @@ export default class HelloClass { say(...things: string[]) { return `hello ${things.join(' and ')}`; } + + reject() { + return Promise.reject(new Error('Rejected')); + } + + throw(message: string) { + throw new Error(message); + } } diff --git a/packages/stryker/test/unit/input/InputFileResolverSpec.ts b/packages/stryker/test/unit/input/InputFileResolverSpec.ts new file mode 100644 index 0000000000..7bb08db86f --- /dev/null +++ b/packages/stryker/test/unit/input/InputFileResolverSpec.ts @@ -0,0 +1,270 @@ +import * as path from 'path'; +import { expect } from 'chai'; +import * as fs from 'mz/fs'; +import * as childProcess from 'mz/child_process'; +import { Logger } from 'log4js'; +import { File } from 'stryker-api/core'; +import { SourceFile } from 'stryker-api/report'; +import InputFileResolver from '../../../src/input/InputFileResolver'; +import * as sinon from 'sinon'; +import * as fileUtils from '../../../src/utils/fileUtils'; +import currentLogMock from '../../helpers/log4jsMock'; +import BroadcastReporter from '../../../src/reporters/BroadcastReporter'; +import { Mock, mock } from '../../helpers/producers'; +import { errorToString, normalizeWhiteSpaces } from '../../../src/utils/objectUtils'; + +const files = (...namesWithContent: [string, string][]): File[] => + namesWithContent.map((nameAndContent): File => new File( + path.resolve(nameAndContent[0]), + Buffer.from(nameAndContent[1]) + )); + +describe('InputFileResolver', () => { + let log: Mock; + let globStub: sinon.SinonStub; + let sut: InputFileResolver; + let reporter: Mock; + let childProcessExecStub: sinon.SinonStub; + let readFileStub: sinon.SinonStub; + + beforeEach(() => { + log = currentLogMock(); + reporter = mock(BroadcastReporter); + globStub = sandbox.stub(fileUtils, 'glob'); + readFileStub = sandbox.stub(fs, 'readFile') + .withArgs(sinon.match.string).resolves(new Buffer(0)) // fallback + .withArgs(sinon.match.string).resolves(new Buffer(0)) // fallback + .withArgs(sinon.match('file1')).resolves(Buffer.from('file 1 content')) + .withArgs(sinon.match('file2')).resolves(Buffer.from('file 2 content')) + .withArgs(sinon.match('file3')).resolves(Buffer.from('file 3 content')) + .withArgs(sinon.match('mute1')).resolves(Buffer.from('mutate 1 content')) + .withArgs(sinon.match('mute2')).resolves(Buffer.from('mutate 2 content')); + globStub.withArgs('mute*').resolves(['/mute1.js', '/mute2.js']); + globStub.withArgs('mute1').resolves(['/mute1.js']); + globStub.withArgs('mute2').resolves(['/mute2.js']); + globStub.withArgs('file1').resolves(['/file1.js']); + globStub.withArgs('file2').resolves(['/file2.js']); + globStub.withArgs('file3').resolves(['/file3.js']); + globStub.withArgs('file*').resolves(['/file1.js', '/file2.js', '/file3.js']); + globStub.resolves([]); // default + childProcessExecStub = sandbox.stub(childProcess, 'exec'); + }); + + it('should use git to identify files if files array is missing', async () => { + sut = new InputFileResolver([], undefined, reporter); + childProcessExecStub.resolves([Buffer.from(` + file1.js + foo/bar/baz.ts + `)]); + const result = await sut.resolve(); + expect(childProcessExecStub).calledWith('git ls-files --others --exclude-standard --cached', + { maxBuffer: 10 * 1000 * 1024 }); + expect(result.files.map(file => file.name)).deep.eq([path.resolve('file1.js'), path.resolve('foo/bar/baz.ts')]); + }); + + it('should reject if there is no `files` array and `git ls-files` command fails', () => { + const expectedError = new Error('fatal: Not a git repository (or any of the parent directories): .git'); + childProcessExecStub.rejects(expectedError); + return expect(new InputFileResolver([], undefined, reporter).resolve()) + .rejectedWith(`Cannot determine input files. Either specify a \`files\` array in your stryker configuration, or make sure "${process.cwd() + }" is located inside a git repository. Inner error: ${ + errorToString(expectedError) + }`); + }); + + it('should log a warning if no files were resolved', async () => { + sut = new InputFileResolver([], [], reporter); + await sut.resolve(); + expect(log.warn).calledWith(sinon.match('No files selected. Please make sure you either run stryker a git repository context')); + expect(log.warn).calledWith(sinon.match('or specify the \`files\` property in your stryker config')); + }); + + it('should be able to handled deleted files reported by `git ls-files`', async () => { + sut = new InputFileResolver([], undefined, reporter); + childProcessExecStub.resolves([Buffer.from(` + deleted/file.js + `)]); + const fileNotFoundError: NodeJS.ErrnoException = new Error(''); + fileNotFoundError.code = 'ENOENT'; + readFileStub.withArgs('deleted/file.js').rejects(fileNotFoundError); + const result = await sut.resolve(); + expect(result.files).lengthOf(0); + }); + + describe('with mutate file expressions', () => { + + it('should result in the expected mutate files', async () => { + sut = new InputFileResolver(['mute*'], ['file1', 'mute1', 'file2', 'mute2', 'file3'], reporter); + const result = await sut.resolve(); + expect(result.filesToMutate.map(_ => _.name)).to.deep.equal([ + path.resolve('/mute1.js'), + path.resolve('/mute2.js') + ]); + expect(result.files.map(file => file.name)).to.deep.equal([ + path.resolve('/file1.js'), + path.resolve('/mute1.js'), + path.resolve('/file2.js'), + path.resolve('/mute2.js'), + path.resolve('/file3.js')] + ); + }); + + it('should only report a mutate file when it is included in the resolved files', async () => { + sut = new InputFileResolver(['mute*'], ['file1', 'mute1', 'file2', /*'mute2'*/ 'file3'], reporter); + const result = await sut.resolve(); + expect(result.filesToMutate.map(_ => _.name)).to.deep.equal([ + path.resolve('/mute1.js') + ]); + }); + + it('should report OnAllSourceFilesRead', async () => { + sut = new InputFileResolver(['mute*'], ['file1', 'mute1', 'file2', 'mute2', 'file3'], reporter); + await sut.resolve(); + const expected: SourceFile[] = [ + { path: path.resolve('/file1.js'), content: 'file 1 content' }, + { path: path.resolve('/mute1.js'), content: 'mutate 1 content' }, + { path: path.resolve('/file2.js'), content: 'file 2 content' }, + { path: path.resolve('/mute2.js'), content: 'mutate 2 content' }, + { path: path.resolve('/file3.js'), content: 'file 3 content' } + ]; + expect(reporter.onAllSourceFilesRead).calledWith(expected); + }); + + it('should report OnSourceFileRead', async () => { + sut = new InputFileResolver(['mute*'], ['file1', 'mute1', 'file2', 'mute2', 'file3'], reporter); + await sut.resolve(); + const expected: SourceFile[] = [ + { path: path.resolve('/file1.js'), content: 'file 1 content' }, + { path: path.resolve('/mute1.js'), content: 'mutate 1 content' }, + { path: path.resolve('/file2.js'), content: 'file 2 content' }, + { path: path.resolve('/mute2.js'), content: 'mutate 2 content' }, + { path: path.resolve('/file3.js'), content: 'file 3 content' } + ]; + expected.forEach(sourceFile => expect(reporter.onSourceFileRead).calledWith(sourceFile)); + }); + }); + + describe('without mutate files', () => { + + beforeEach(() => { + sut = new InputFileResolver([], ['file1', 'mute1'], reporter); + }); + + it('should warn about dry-run', async () => { + await sut.resolve(); + expect(log.warn).to.have.been.calledWith('No files marked to be mutated, stryker will perform a dry-run without actually mutating anything.'); + }); + }); + + describe('with file expressions that resolve in different order', () => { + beforeEach(() => { + sut = new InputFileResolver([], ['fileWhichResolvesLast', 'fileWhichResolvesFirst'], reporter); + globStub.withArgs('fileWhichResolvesLast').resolves(['file1']); + globStub.withArgs('fileWhichResolvesFirst').resolves(['file2']); + }); + + it('should retain original glob order', async () => { + const result = await sut.resolve(); + expect(result.files.map(m => m.name.substr(m.name.length - 5))).to.deep.equal(['file1', 'file2']); + }); + }); + + describe('with file expressions in the old `InputFileDescriptor` syntax', () => { + let patternFile1: any; + let patternFile3: any; + + beforeEach(() => { + patternFile1 = { pattern: 'file1' }; + patternFile3 = { pattern: 'file3' }; + + sut = new InputFileResolver([], [patternFile1, 'file2', patternFile3], reporter); + }); + + it('it should log a warning', async () => { + + await sut.resolve(); + const inputFileDescriptors = JSON.stringify([patternFile1, patternFile3]); + const patternNames = JSON.stringify([patternFile1.pattern, patternFile3.pattern]); + expect(log.warn).calledWith(normalizeWhiteSpaces(` + DEPRECATED: Using the \`InputFileDescriptor\` syntax to + select files is no longer supported. We'll assume: ${inputFileDescriptors} can be migrated + to ${patternNames} for this mutation run. Please move any files to mutate into the \`mutate\` + array (top level stryker option). + + You can fix this warning in 2 ways: + 1) If your project is under git version control, you can remove the "files" patterns all together. + Stryker can figure it out for you. + 2) If your project is not under git version control or you need ignored files in your sandbox, you can replace the + \`InputFileDescriptor\` syntax with strings (as done for this test run).`)); + }); + + it('should resolve the patterns as normal files', async () => { + const result = await sut.resolve(); + const actualFileNames = result.files.map(m => m.name); + expect(actualFileNames).to.deep.equal(['/file1.js', '/file2.js', '/file3.js'].map(_ => path.resolve(_))); + }); + }); + + describe('when a globbing expression does not result in a result', () => { + beforeEach(() => { + sut = new InputFileResolver(['file1'], ['file1', 'notExists'], reporter); + }); + + it('should log a warning', async () => { + await sut.resolve(); + expect(log.warn).to.have.been.calledWith('Globbing expression "notExists" did not result in any files.'); + }); + }); + + it('should reject when a globbing expression results in a reject', () => { + sut = new InputFileResolver(['file1'], ['fileError', 'fileError'], reporter); + const expectedError = new Error('ERROR: something went wrong'); + globStub.withArgs('fileError').rejects(expectedError); + return expect(sut.resolve()).rejectedWith(expectedError); + }); + + describe('when excluding files with "!"', () => { + + it('should exclude the files that were previously included', async () => { + const result = await new InputFileResolver([], ['file2', 'file1', '!file2'], reporter).resolve(); + assertFilesEqual(result.files, files(['/file1.js', 'file 1 content'])); + }); + + it('should exclude the files that were previously with a wild card', async () => { + const result = await new InputFileResolver([], ['file*', '!file2'], reporter).resolve(); + assertFilesEqual(result.files, files(['/file1.js', 'file 1 content'], ['/file3.js', 'file 3 content'])); + }); + + it('should not exclude files when the globbing expression results in an empty array', async () => { + const result = await new InputFileResolver([], ['file2', '!does/not/exist'], reporter).resolve(); + assertFilesEqual(result.files, files(['/file2.js', 'file 2 content'])); + }); + }); + + describe('when provided duplicate files', () => { + + it('should deduplicate files that occur more than once', async () => { + const result = await new InputFileResolver([], ['file2', 'file2'], reporter).resolve(); + assertFilesEqual(result.files, files(['/file2.js', 'file 2 content'])); + }); + + it('should deduplicate files that previously occurred in a wildcard expression', async () => { + const result = await new InputFileResolver([], ['file*', 'file2'], reporter).resolve(); + assertFilesEqual(result.files, files(['/file1.js', 'file 1 content'], ['/file2.js', 'file 2 content'], ['/file3.js', 'file 3 content'])); + }); + + it('should order files by expression order', async () => { + const result = await new InputFileResolver([], ['file2', 'file*'], reporter).resolve(); + assertFilesEqual(result.files, files(['/file2.js', 'file 2 content'], ['/file1.js', 'file 1 content'], ['/file3.js', 'file 3 content'])); + }); + }); + + function assertFilesEqual(actual: ReadonlyArray, expected: ReadonlyArray) { + expect(actual).lengthOf(expected.length); + for (let index in actual) { + expect(actual[index].name).eq(expected[index].name); + expect(actual[index].textContent).eq(expected[index].textContent); + } + } + +}); \ No newline at end of file diff --git a/packages/stryker/test/unit/isolated-runner/IsolatedTestRunnerAdapterSpec.ts b/packages/stryker/test/unit/isolated-runner/IsolatedTestRunnerAdapterSpec.ts index 88af7d9cb4..03837a2154 100644 --- a/packages/stryker/test/unit/isolated-runner/IsolatedTestRunnerAdapterSpec.ts +++ b/packages/stryker/test/unit/isolated-runner/IsolatedTestRunnerAdapterSpec.ts @@ -23,8 +23,8 @@ describe('IsolatedTestRunnerAdapter', () => { beforeEach(() => { runnerOptions = { + fileNames: [], port: 42, - files: [], sandboxWorkingFolder: 'a working directory', strykerOptions: {} }; @@ -71,7 +71,7 @@ describe('IsolatedTestRunnerAdapter', () => { it(' "initDone"', () => { arrangeAct(); - receiveMessage({ kind: 'initDone' }); + receiveMessage({ kind: 'initDone', errorMessage: null }); return expect(initPromise).to.eventually.eq(undefined); }); diff --git a/packages/stryker/test/unit/mutators/ES5MutantGeneratorSpec.ts b/packages/stryker/test/unit/mutators/ES5MutantGeneratorSpec.ts index e6fa4f6703..5a33d12405 100644 --- a/packages/stryker/test/unit/mutators/ES5MutantGeneratorSpec.ts +++ b/packages/stryker/test/unit/mutators/ES5MutantGeneratorSpec.ts @@ -2,10 +2,10 @@ import { expect } from 'chai'; import { Syntax } from 'esprima'; import * as estree from 'estree'; import { Mutant } from 'stryker-api/mutant'; -import { file, textFile } from '../../helpers/producers'; import ES5Mutator from '../../../src/mutators/ES5Mutator'; import NodeMutator from '../../../src/mutators/NodeMutator'; import { Identified, IdentifiedNode } from '../../../src/mutators/IdentifiedNode'; +import { File } from 'stryker-api/core'; describe('ES5Mutator', () => { let sut: ES5Mutator; @@ -18,16 +18,11 @@ describe('ES5Mutator', () => { sandbox.restore(); }); - it('should return an empty array if nothing could be mutated', () => { - const mutants = sut.mutate([textFile({ name: 'test.js', included: false, mutated: true, content: '' })]); - expect(mutants.length).to.equal(0); - }); - describe('with single input file with a one possible mutation', () => { let mutants: Mutant[]; beforeEach(() => { - mutants = sut.mutate([file({ content: 'var i = 1 + 2;' })]); + mutants = sut.mutate([new File('', 'var i = 1 + 2;')]); }); it('should return an array with a single mutant', () => { @@ -40,7 +35,7 @@ describe('ES5Mutator', () => { it('should set the range', () => { const originalCode = '\n\nvar i = 1 + 2;'; - mutants = sut.mutate([file({ content: originalCode })]); + mutants = sut.mutate([new File('', originalCode)]); expect(mutants[0].range[0]).to.equal(10); expect(mutants[0].range[1]).to.equal(15); }); @@ -68,13 +63,13 @@ describe('ES5Mutator', () => { }); it('the same nodeID', () => { - const mutants = sut.mutate([file({ name: 'some file', content: 'if (true);' })]); + const mutants = sut.mutate([new File('some file', 'if (true);')]); expect(mutants[0].fileName).eq('some file'); expect(mutants[0].replacement).eq('if (true);'); }); it('a different nodeID', () => { - const mutants = sut.mutate([file({ name: 'src.js', content: '1 * 2' })]); + const mutants = sut.mutate([new File('src.js', '1 * 2')]); expect(mutants[0].fileName).eq('src.js'); expect(mutants[0].replacement).eq('1'); }); diff --git a/packages/stryker/test/unit/process/InitialTestExecutorSpec.ts b/packages/stryker/test/unit/process/InitialTestExecutorSpec.ts index 9c11596b76..31ff0259a7 100644 --- a/packages/stryker/test/unit/process/InitialTestExecutorSpec.ts +++ b/packages/stryker/test/unit/process/InitialTestExecutorSpec.ts @@ -9,12 +9,16 @@ import * as producers from '../../helpers/producers'; import { TestFramework } from 'stryker-api/test_framework'; import CoverageInstrumenterTranspiler, * as coverageInstrumenterTranspiler from '../../../src/transpiler/CoverageInstrumenterTranspiler'; import TranspilerFacade, * as transpilerFacade from '../../../src/transpiler/TranspilerFacade'; -import { TranspileResult, TranspilerOptions } from 'stryker-api/transpile'; +import { TranspilerOptions } from 'stryker-api/transpile'; import { RunStatus, RunResult, TestStatus } from 'stryker-api/test_runner'; import currentLogMock from '../../helpers/log4jsMock'; import Timer from '../../../src/utils/Timer'; import { Mock, coverageMaps } from '../../helpers/producers'; +import InputFileCollection from '../../../src/input/InputFileCollection'; +import * as coverageHooks from '../../../src/transpiler/coverageHooks'; +import SourceMapper, { PassThroughSourceMapper } from '../../../src/transpiler/SourceMapper'; +const EXPECTED_INITIAL_TIMEOUT = 60 * 1000 * 5; describe('InitialTestExecutor run', () => { let log: Mock; @@ -24,7 +28,9 @@ describe('InitialTestExecutor run', () => { let coverageInstrumenterTranspilerMock: producers.Mock; let options: Config; let transpilerFacadeMock: producers.Mock; - let transpileResultMock: TranspileResult; + let transpiledFiles: File[]; + let coverageAnnotatedFiles: File[]; + let sourceMapperMock: producers.Mock; let timer: producers.Mock; let expectedRunResult: RunResult; @@ -36,47 +42,43 @@ describe('InitialTestExecutor run', () => { sandbox.stub(StrykerSandbox, 'create').resolves(strykerSandboxMock); sandbox.stub(transpilerFacade, 'default').returns(transpilerFacadeMock); sandbox.stub(coverageInstrumenterTranspiler, 'default').returns(coverageInstrumenterTranspilerMock); + sourceMapperMock = producers.mock(PassThroughSourceMapper); + sandbox.stub(SourceMapper, 'create').returns(sourceMapperMock); testFrameworkMock = producers.testFramework(); - transpileResultMock = producers.transpileResult({ - outputFiles: [ - producers.textFile({ name: 'transpiled-file-1.js' }), - producers.textFile({ name: 'transpiled-file-2.js' }) - ] - }); - transpilerFacadeMock.transpile.returns(transpileResultMock); + coverageAnnotatedFiles = [ + new File('cov-annotated-transpiled-file-1.js', ''), + new File('cov-annotated-transpiled-file-2.js', ''), + ]; + transpiledFiles = [ + new File('transpiled-file-1.js', ''), + new File('transpiled-file-2.js', '') + ]; + coverageInstrumenterTranspilerMock.transpile.returns(coverageAnnotatedFiles); + transpilerFacadeMock.transpile.returns(transpiledFiles); options = producers.config(); expectedRunResult = producers.runResult(); strykerSandboxMock.run.resolves(expectedRunResult); timer = producers.mock(Timer); }); - describe('without input files', () => { - it('should log a warning and cancel the test run', async () => { - sut = new InitialTestExecutor(options, [], testFrameworkMock, timer as any); - const result = await sut.run(); - expect(result.runResult.status).to.be.eq(RunStatus.Complete); - expect(log.info).to.have.been.calledWith('No files have been found. Aborting initial test run.'); - }); - }); - describe('with input files', () => { - let files: File[]; + let inputFiles: InputFileCollection; beforeEach(() => { - files = [producers.textFile({ name: '', mutated: true, included: true, content: '' })]; - sut = new InitialTestExecutor(options, files, testFrameworkMock, timer as any); + inputFiles = new InputFileCollection([new File('mutate.js', ''), new File('mutate.spec.js', '')], ['mutate.js']); + sut = new InitialTestExecutor(options, inputFiles, testFrameworkMock, timer as any); }); it('should create a sandbox with correct arguments', async () => { await sut.run(); - expect(StrykerSandbox.create).calledWith(options, 0, transpileResultMock.outputFiles, testFrameworkMock); + expect(StrykerSandbox.create).calledWith(options, 0, coverageAnnotatedFiles, testFrameworkMock); }); it('should create the transpiler with produceSourceMaps = true when coverage analysis is enabled', async () => { options.coverageAnalysis = 'all'; await sut.run(); - const expectedTranspilerOptions: TranspilerOptions = { + const expectedTranspilerOptions: TranspilerOptions = { produceSourceMaps: true, config: options }; @@ -87,7 +89,7 @@ describe('InitialTestExecutor run', () => { it('should create the transpiler with produceSourceMaps = false when coverage analysis is "off"', async () => { options.coverageAnalysis = 'off'; await sut.run(); - const expectedTranspilerOptions: TranspilerOptions = { + const expectedTranspilerOptions: TranspilerOptions = { produceSourceMaps: false, config: options }; @@ -96,7 +98,7 @@ describe('InitialTestExecutor run', () => { it('should initialize, run and dispose the sandbox', async () => { await sut.run(); - expect(strykerSandboxMock.run).to.have.been.calledWith(60 * 1000 * 5); + expect(strykerSandboxMock.run).to.have.been.calledWith(EXPECTED_INITIAL_TIMEOUT); expect(strykerSandboxMock.dispose).to.have.been.called; }); @@ -105,7 +107,7 @@ describe('InitialTestExecutor run', () => { coverageInstrumenterTranspilerMock.fileCoverageMaps = { someFile: coverageData } as any; const expectedResult: InitialTestRunResult = { runResult: expectedRunResult, - transpiledFiles: transpileResultMock.outputFiles, + sourceMapper: sourceMapperMock, coverageMaps: { someFile: coverageData } @@ -114,28 +116,19 @@ describe('InitialTestExecutor run', () => { expect(actualRunResult).deep.eq(expectedResult); }); - it('should log the transpiled results if transpilers are specified and log.debug is enabled', async () => { + it('should log the transpiled results if transpilers are specified', async () => { options.transpilers.push('a transpiler'); log.isDebugEnabled.returns(true); await sut.run(); - expect(log.debug).calledOnce; const actualLogMessage: string = log.debug.getCall(0).args[0]; - expect(actualLogMessage).contains('Transpiled files in order'); - expect(actualLogMessage).contains('transpiled-file-1.js (included: true)'); - expect(actualLogMessage).contains('transpiled-file-2.js (included: true)'); + const expectedLogMessage = `Transpiled files: ${JSON.stringify(coverageAnnotatedFiles.map(_ => _.name), null, 2)}`; + expect(actualLogMessage).eq(expectedLogMessage); }); it('should not log the transpiled results if transpilers are not specified', async () => { log.isDebugEnabled.returns(true); await sut.run(); - expect(log.debug).not.called; - }); - - it('should not log the transpiled results if log.debug is disabled', async () => { - options.transpilers.push('a transpiler'); - log.isDebugEnabled.returns(false); - await sut.run(); - expect(log.debug).not.called; + expect(log.debug).not.calledWithMatch('Transpiled files'); }); it('should have logged the amount of tests ran', async () => { @@ -156,16 +149,26 @@ describe('InitialTestExecutor run', () => { await expect(sut.run()).rejectedWith(expectedError); }); - it('should add the coverage instrumenter transpiler', async () => { + it('should create the coverage instrumenter transpiler with source-mapped files to mutate', async () => { + sourceMapperMock.transpiledFileNameFor.returns('mutate.min.js'); await sut.run(); - const expectedSettings: TranspilerOptions = { - config: options, - produceSourceMaps: true - }; expect(coverageInstrumenterTranspiler.default).calledWithNew; - expect(coverageInstrumenterTranspiler.default).calledWith(expectedSettings, testFrameworkMock); + expect(coverageInstrumenterTranspiler.default).calledWith(options, ['mutate.min.js']); + expect(sourceMapperMock.transpiledFileNameFor).calledWith('mutate.js'); + }); + + it('should also add a collectCoveragePerTest file when coverage analysis is "perTest" and there is a testFramework', async () => { + sandbox.stub(coverageHooks, 'coveragePerTestHooks').returns('test hook foobar'); + await sut.run(); + expect(strykerSandboxMock.run).calledWith(EXPECTED_INITIAL_TIMEOUT, 'test hook foobar'); }); + it('should result log a warning if coverage analysis is "perTest" and there is no testFramework', async () => { + sut = new InitialTestExecutor(options, inputFiles, /* test framework */ null, timer as any); + sandbox.stub(coverageHooks, 'coveragePerTestHooks').returns('test hook foobar'); + await sut.run(); + expect(log.warn).calledWith('Cannot measure coverage results per test, there is no testFramework and thus no way of executing code right before and after each test.'); + }); describe('and run has test failures', () => { beforeEach(() => { @@ -218,8 +221,6 @@ describe('InitialTestExecutor run', () => { await expect(sut.run()).rejectedWith('Something went wrong in the initial test run'); }); }); - }); - }); diff --git a/packages/stryker/test/unit/process/MutationTestExecutorSpec.ts b/packages/stryker/test/unit/process/MutationTestExecutorSpec.ts index 5b2c374325..51621040ce 100644 --- a/packages/stryker/test/unit/process/MutationTestExecutorSpec.ts +++ b/packages/stryker/test/unit/process/MutationTestExecutorSpec.ts @@ -5,7 +5,6 @@ import { Config } from 'stryker-api/config'; import { File } from 'stryker-api/core'; import { TestFramework } from 'stryker-api/test_framework'; import { RunStatus, TestStatus } from 'stryker-api/test_runner'; -import { TranspileResult } from 'stryker-api/transpile'; import Sandbox from '../../../src/Sandbox'; import BroadcastReporter from '../../../src/reporters/BroadcastReporter'; import MutantTestExecutor from '../../../src/process/MutationTestExecutor'; @@ -13,7 +12,7 @@ import TranspiledMutant from '../../../src/TranspiledMutant'; import { MutantStatus } from 'stryker-api/report'; import MutantTranspiler, * as mutantTranspiler from '../../../src/transpiler/MutantTranspiler'; import SandboxPool, * as sandboxPool from '../../../src/SandboxPool'; -import { transpiledMutant, testResult, Mock, mock, textFile, config, testFramework, testableMutant, mutantResult, transpileResult } from '../../helpers/producers'; +import { transpiledMutant, testResult, Mock, mock, config, testFramework, testableMutant, mutantResult, file } from '../../helpers/producers'; import '../../helpers/globals'; import TestableMutant from '../../../src/TestableMutant'; @@ -38,19 +37,19 @@ describe('MutationTestExecutor', () => { let expectedConfig: Config; let sut: MutantTestExecutor; let mutants: TestableMutant[]; - let initialTranspileResult: TranspileResult; + let initialTranspiledFiles: File[]; beforeEach(() => { sandboxPoolMock = mock(SandboxPool); mutantTranspilerMock = mock(MutantTranspiler); - initialTranspileResult = transpileResult({ outputFiles: [textFile(), textFile()] }); - mutantTranspilerMock.initialize.resolves(initialTranspileResult); + initialTranspiledFiles = [file(), file()]; + mutantTranspilerMock.initialize.resolves(initialTranspiledFiles); sandboxPoolMock.disposeAll.resolves(); testFrameworkMock = testFramework(); sandbox.stub(sandboxPool, 'default').returns(sandboxPoolMock); sandbox.stub(mutantTranspiler, 'default').returns(mutantTranspilerMock); reporter = mock(BroadcastReporter); - inputFiles = [textFile({ name: 'input.ts' })]; + inputFiles = [new File('input.ts', '')]; expectedConfig = config(); mutants = [testableMutant()]; }); @@ -71,7 +70,7 @@ describe('MutationTestExecutor', () => { expect(mutantTranspiler.default).calledWithNew; }); it('should create the sandbox pool', () => { - expect(sandboxPool.default).calledWith(expectedConfig, testFrameworkMock, initialTranspileResult.outputFiles); + expect(sandboxPool.default).calledWith(expectedConfig, testFrameworkMock, initialTranspiledFiles); expect(sandboxPool.default).calledWithNew; }); diff --git a/packages/stryker/test/unit/transpiler/CoverageInstrumenterTranspilerSpec.ts b/packages/stryker/test/unit/transpiler/CoverageInstrumenterTranspilerSpec.ts index 58dec2e518..203e661a85 100644 --- a/packages/stryker/test/unit/transpiler/CoverageInstrumenterTranspilerSpec.ts +++ b/packages/stryker/test/unit/transpiler/CoverageInstrumenterTranspilerSpec.ts @@ -1,8 +1,7 @@ import { expect } from 'chai'; import { Config } from 'stryker-api/config'; -import { TextFile } from 'stryker-api/core'; import CoverageInstrumenterTranspiler from '../../../src/transpiler/CoverageInstrumenterTranspiler'; -import { textFile, binaryFile, webFile, testFramework } from '../../helpers/producers'; +import { File } from 'stryker-api/core'; describe('CoverageInstrumenterTranspiler', () => { let sut: CoverageInstrumenterTranspiler; @@ -13,98 +12,54 @@ describe('CoverageInstrumenterTranspiler', () => { }); it('should not instrument any code when coverage analysis is off', async () => { - sut = new CoverageInstrumenterTranspiler({ config, produceSourceMaps: false }, null); + sut = new CoverageInstrumenterTranspiler(config, ['foobar.js']); config.coverageAnalysis = 'off'; - const input = [textFile({ mutated: true }), binaryFile({ mutated: true }), webFile({ mutated: true })]; - const output = await sut.transpile(input); - expect(output.error).null; - expect(output.outputFiles).deep.eq(input); + const input = [new File('foobar.js', '')]; + const outputFiles = await sut.transpile(input); + expect(outputFiles).deep.eq(input); }); describe('when coverage analysis is "all"', () => { beforeEach(() => { config.coverageAnalysis = 'all'; - sut = new CoverageInstrumenterTranspiler({ config, produceSourceMaps: false }, null); + sut = new CoverageInstrumenterTranspiler(config, ['mutate.js']); }); it('should instrument code of mutated files', async () => { const input = [ - textFile({ mutated: true, content: 'function something() {}' }), - binaryFile({ mutated: true }), - webFile({ mutated: true }), - textFile({ mutated: false }) + new File('mutate.js', 'function something() {}'), + new File('spec.js', '') ]; - const output = await sut.transpile(input); - expect(output.error).null; - expect(output.outputFiles[1]).eq(output.outputFiles[1]); - expect(output.outputFiles[2]).eq(output.outputFiles[2]); - expect(output.outputFiles[3]).eq(output.outputFiles[3]); - const instrumentedContent = (output.outputFiles[0] as TextFile).content; + const outputFiles = await sut.transpile(input); + const instrumentedContent = outputFiles[0].textContent; expect(instrumentedContent).to.contain('function something(){cov_').and.contain('.f[0]++'); }); - it('should preserve source map comments', async () => { + it('should preserve source map comments', async () => { const input = [ - textFile({ mutated: true, content: 'function something() {} // # sourceMappingUrl="something.map.js"' }), + new File('mutate.js', 'function something() {} // # sourceMappingUrl="something.map.js"'), ]; - const output = await sut.transpile(input); - expect(output.error).null; - const instrumentedContent = (output.outputFiles[0] as TextFile).content; + const outputFiles = await sut.transpile(input); + const instrumentedContent = outputFiles[0].textContent; expect(instrumentedContent).to.contain('sourceMappingUrl="something.map.js"'); }); it('should create a statement map for mutated files', () => { const input = [ - textFile({ name: 'something.js', mutated: true, content: 'function something () {}' }), - textFile({ name: 'foobar.js', mutated: true, content: 'console.log("foobar");' }) + new File('mutate.js', 'function something () {}'), + new File('foobar.js', 'console.log("foobar");') ]; sut.transpile(input); - expect(sut.fileCoverageMaps['something.js'].statementMap).deep.eq({}); - expect(sut.fileCoverageMaps['something.js'].fnMap[0]).deep.eq({ start: { line: 0, column: 22 }, end: { line: 0, column: 24 } }); - expect(sut.fileCoverageMaps['something.js'].fnMap[1]).undefined; - expect(sut.fileCoverageMaps['foobar.js'].statementMap).deep.eq({ '0': { start: { line: 0, column: 0 }, end: { line: 0, column: 22 } } }); - expect(sut.fileCoverageMaps['foobar.js'].fnMap).deep.eq({}); + expect(sut.fileCoverageMaps['mutate.js'].statementMap).deep.eq({}); + expect(sut.fileCoverageMaps['mutate.js'].fnMap[0]).deep.eq({ start: { line: 0, column: 22 }, end: { line: 0, column: 24 } }); + expect(sut.fileCoverageMaps['mutate.js'].fnMap[1]).undefined; + expect(sut.fileCoverageMaps['foobar.js']).undefined; }); it('should fill error message and not transpile input when the file contains a parse error', async () => { - const invalidJavascriptFile = textFile({ name: 'invalid/file.js', content: 'function something {}', mutated: true }); - const output = await sut.transpile([invalidJavascriptFile]); - expect(output.error).contains('Could not instrument "invalid/file.js" for code coverage. SyntaxError: Unexpected token'); + const invalidJavascriptFile = new File('mutate.js', 'function something {}'); + return expect(sut.transpile([invalidJavascriptFile])).rejectedWith('Could not instrument "mutate.js" for code coverage. Inner error: SyntaxError: Unexpected token'); }); }); - - describe('when coverage analysis is "perTest" and there is a testFramework', () => { - let input: TextFile[]; - - beforeEach(() => { - config.coverageAnalysis = 'perTest'; - sut = new CoverageInstrumenterTranspiler({ config, produceSourceMaps: false }, testFramework()); - input = [textFile({ mutated: true, content: 'function something() {}' })]; - }); - - it('should use the coverage variable "__strykerCoverageCurrentTest__"', async () => { - const output = await sut.transpile(input); - expect(output.error).null; - const instrumentedContent = (output.outputFiles[1] as TextFile).content; - expect(instrumentedContent).to.contain('__strykerCoverageCurrentTest__').and.contain('.f[0]++'); - }); - - it('should also add a collectCoveragePerTest file', async () => { - const output = await sut.transpile(input); - expect(output.error).null; - expect(output.outputFiles).lengthOf(2); - const actualContent = (output.outputFiles[0] as TextFile).content; - expect(actualContent).to.have.length.greaterThan(30); - expect(actualContent).to.contain('beforeEach()'); - expect(actualContent).to.contain('afterEach()'); - }); - }); - - it('should result in an error if coverage analysis is "perTest" and there is no testFramework', async () => { - config.coverageAnalysis = 'perTest'; - sut = new CoverageInstrumenterTranspiler({ config, produceSourceMaps: true }, null); - const output = await sut.transpile([textFile({ content: 'a + b' })]); - expect(output.error).eq('Cannot measure coverage results per test, there is no testFramework and thus no way of executing code right before and after each test.'); - }); }); \ No newline at end of file diff --git a/packages/stryker/test/unit/transpiler/MutantTranspilerSpec.ts b/packages/stryker/test/unit/transpiler/MutantTranspilerSpec.ts index 9165edc758..2cc8d27a7d 100644 --- a/packages/stryker/test/unit/transpiler/MutantTranspilerSpec.ts +++ b/packages/stryker/test/unit/transpiler/MutantTranspilerSpec.ts @@ -2,16 +2,18 @@ import { expect } from 'chai'; import MutantTranspiler from '../../../src/transpiler/MutantTranspiler'; import ChildProcessProxy from '../../../src/child-proxy/ChildProcessProxy'; import TranspilerFacade, * as transpilerFacade from '../../../src/transpiler/TranspilerFacade'; -import { Mock, mock, transpileResult, config, textFile, webFile, testableMutant } from '../../helpers/producers'; -import { TranspileResult } from 'stryker-api/transpile'; +import { Mock, mock, config, testableMutant, file } from '../../helpers/producers'; import '../../helpers/globals'; import TranspiledMutant from '../../../src/TranspiledMutant'; +import TranspileResult from '../../../src/transpiler/TranspileResult'; +import { File } from 'stryker-api/core'; +import { errorToString } from '../../../src/utils/objectUtils'; describe('MutantTranspiler', () => { let sut: MutantTranspiler; let transpilerFacadeMock: Mock; - let transpileResultOne: TranspileResult; - let transpileResultTwo: TranspileResult; + let transpiledFilesOne: File[]; + let transpiledFilesTwo: File[]; let childProcessProxyMock: { proxy: Mock, dispose: sinon.SinonStub }; beforeEach(() => { @@ -19,11 +21,11 @@ describe('MutantTranspiler', () => { childProcessProxyMock = { proxy: transpilerFacadeMock, dispose: sandbox.stub() }; sandbox.stub(ChildProcessProxy, 'create').returns(childProcessProxyMock); sandbox.stub(transpilerFacade, 'default').returns(transpilerFacadeMock); - transpileResultOne = transpileResult({ error: 'first result' }); - transpileResultTwo = transpileResult({ error: 'second result' }); + transpiledFilesOne = [new File('firstResult.js', 'first result')]; + transpiledFilesTwo = [new File('secondResult.js', 'second result')]; transpilerFacadeMock.transpile - .onFirstCall().resolves(transpileResultOne) - .onSecondCall().resolves(transpileResultTwo); + .onFirstCall().resolves(transpiledFilesOne) + .onSecondCall().resolves(transpiledFilesTwo); }); describe('with a transpiler', () => { @@ -42,11 +44,11 @@ describe('MutantTranspiler', () => { describe('initialize', () => { it('should transpile all files', () => { - const expectedFiles = [textFile(), webFile()]; + const expectedFiles = [file()]; sut = new MutantTranspiler(config({ transpilers: ['transpiler'] })); const actualResult = sut.initialize(expectedFiles); expect(transpilerFacadeMock.transpile).calledWith(expectedFiles); - return expect(actualResult).eventually.eq(transpileResultOne); + return expect(actualResult).eventually.eq(transpiledFilesOne); }); }); @@ -70,8 +72,27 @@ describe('MutantTranspiler', () => { .toArray() .toPromise(); const expected: TranspiledMutant[] = [ - { mutant: mutants[0], transpileResult: transpileResultOne, changedAnyTranspiledFiles: true }, - { mutant: mutants[1], transpileResult: transpileResultTwo, changedAnyTranspiledFiles: true } + { mutant: mutants[0], transpileResult: { error: null, outputFiles: transpiledFilesOne }, changedAnyTranspiledFiles: true }, + { mutant: mutants[1], transpileResult: { error: null, outputFiles: transpiledFilesTwo }, changedAnyTranspiledFiles: true } + ]; + expect(actualResult).deep.eq(expected); + }); + + it('should report rejected transpile attempts as errors', async () => { + // Arrange + const error = new Error('expected transpile error'); + transpilerFacadeMock.transpile.reset(); + transpilerFacadeMock.transpile.rejects(error); + const mutant = testableMutant(); + + // Act + const actualResult = await sut.transpileMutants([mutant]) + .toArray() + .toPromise(); + + // Assert + const expected: TranspiledMutant[] = [ + { mutant, transpileResult: { error: errorToString(error), outputFiles: [] }, changedAnyTranspiledFiles: false }, ]; expect(actualResult).deep.eq(expected); }); @@ -79,9 +100,9 @@ describe('MutantTranspiler', () => { it('should set set the changedAnyTranspiledFiles boolean to false if transpiled output did not change', async () => { // Arrange transpilerFacadeMock.transpile.reset(); - transpilerFacadeMock.transpile.resolves(transpileResultOne); + transpilerFacadeMock.transpile.resolves(transpiledFilesOne); const mutants = [testableMutant()]; - const files = [textFile()]; + const files = [file()]; await sut.initialize(files); // Act @@ -91,37 +112,41 @@ describe('MutantTranspiler', () => { // Assert const expected: TranspiledMutant[] = [ - { mutant: mutants[0], transpileResult: transpileResultOne, changedAnyTranspiledFiles: false } + { mutant: mutants[0], transpileResult: { error: null, outputFiles: transpiledFilesOne }, changedAnyTranspiledFiles: false } ]; expect(actual).deep.eq(expected); }); it('should transpile mutants one by one in sequence', async () => { // Arrange - let resolveFirst: (transpileResult: TranspileResult) => void = () => { }; - let resolveSecond: (transpileResult: TranspileResult) => void = () => { }; + let resolveFirst: (files: ReadonlyArray) => void = () => { }; + let resolveSecond: (files: ReadonlyArray) => void = () => { }; transpilerFacadeMock.transpile.resetBehavior(); transpilerFacadeMock.transpile - .onFirstCall().returns(new Promise(res => resolveFirst = res)) - .onSecondCall().returns(new Promise(res => resolveSecond = res)); + .onFirstCall().returns(new Promise>(res => resolveFirst = res)) + .onSecondCall().returns(new Promise>(res => resolveSecond = res)); const actualResults: TranspileResult[] = []; // Act sut.transpileMutants([testableMutant('one'), testableMutant('two')]) - .subscribe(transpileResult => actualResults.push(transpileResult.transpileResult)); + .subscribe(transpiledMutant => actualResults.push(transpiledMutant.transpileResult)); // Assert: only first time called expect(transpilerFacadeMock.transpile).calledOnce; expect(actualResults).lengthOf(0); - resolveFirst(transpileResultOne); + resolveFirst(transpiledFilesOne); await nextTick(); // Assert: second one is called, first one is received expect(transpilerFacadeMock.transpile).calledTwice; expect(actualResults).lengthOf(1); - resolveSecond(transpileResultTwo); + resolveSecond(transpiledFilesTwo); // Assert: all results are in await nextTick(); - expect(actualResults).deep.eq([transpileResultOne, transpileResultTwo]); + const expectedResults: TranspileResult[] = [ + { error: null, outputFiles: transpiledFilesOne }, + { error: null, outputFiles: transpiledFilesTwo } + ]; + expect(actualResults).deep.eq(expectedResults); }); const nextTick = () => new Promise(res => { setTimeout(res, 0); @@ -141,20 +166,19 @@ describe('MutantTranspiler', () => { }); it('should transpile the files when initialized', async () => { - const expectedFiles = [textFile(), webFile()]; + const expectedFiles = [file()]; const actualFiles = await sut.initialize(expectedFiles); expect(transpilerFacadeMock.transpile).calledWith(expectedFiles); - expect(actualFiles).eq(transpileResultOne); - expect(actualFiles.error).eq('first result'); + expect(actualFiles).eq(transpiledFilesOne); }); it('should transpile the mutated files when transpileMutants is called', async () => { const actualMutants = [testableMutant('file1.ts'), testableMutant('file2.ts')]; const actualResult = await sut.transpileMutants(actualMutants).toArray().toPromise(); expect(actualResult).lengthOf(2); - expect(actualResult[0].transpileResult).eq(transpileResultOne); + expect(actualResult[0].transpileResult.outputFiles).eq(transpiledFilesOne); expect(actualResult[0].mutant).eq(actualMutants[0]); - expect(actualResult[1].transpileResult).eq(transpileResultTwo); + expect(actualResult[1].transpileResult.outputFiles).eq(transpiledFilesTwo); expect(actualResult[1].mutant).eq(actualMutants[1]); }); diff --git a/packages/stryker/test/unit/transpiler/SourceMapperSpec.ts b/packages/stryker/test/unit/transpiler/SourceMapperSpec.ts index 2f31e8d1e0..9e0f5a4410 100644 --- a/packages/stryker/test/unit/transpiler/SourceMapperSpec.ts +++ b/packages/stryker/test/unit/transpiler/SourceMapperSpec.ts @@ -3,7 +3,7 @@ import * as sourceMapModule from 'source-map'; import { Config } from 'stryker-api/config'; import { File } from 'stryker-api/core'; import SourceMapper, { PassThroughSourceMapper, TranspiledSourceMapper, MappedLocation, SourceMapError } from '../../../src/transpiler/SourceMapper'; -import { Mock, mock, config as configFactory, location as locationFactory, textFile, mappedLocation, binaryFile } from '../../helpers/producers'; +import { Mock, mock, config as configFactory, location as locationFactory, mappedLocation, PNG_BASE64_ENCODED } from '../../helpers/producers'; const GREATEST_LOWER_BOUND = sourceMapModule.SourceMapConsumer.GREATEST_LOWER_BOUND; const LEAST_UPPER_BOUND = sourceMapModule.SourceMapConsumer.LEAST_UPPER_BOUND; @@ -70,19 +70,13 @@ describe('SourceMapper', () => { sut = new TranspiledSourceMapper(transpiledFiles); }); - it('should create SourceMapConsumers for mutated text files when transpiledLocationFor is called', () => { + it('should create SourceMapConsumers for files when transpiledLocationFor is called', () => { // Arrange const expectedMapFile1 = { sources: ['file1.ts'] }; const expectedMapFile2 = { sources: ['file2.ts'] }; - transpiledFiles.push(textFile({ - name: 'file1.js', mutated: true, content: `// # sourceMappingURL=file1.js.map` - })); - transpiledFiles.push(textFile({ - name: 'file1.js.map', mutated: false, content: JSON.stringify(expectedMapFile1) - })); - transpiledFiles.push(textFile({ - name: 'file2.js', mutated: true, content: `// # sourceMappingURL=data:application/json;base64,${base64Encode(JSON.stringify(expectedMapFile2))}` - })); + transpiledFiles.push(new File('file1.js', '// # sourceMappingURL=file1.js.map')); + transpiledFiles.push(new File('file1.js.map', JSON.stringify(expectedMapFile1))); + transpiledFiles.push(new File('file2.js', `// # sourceMappingURL=data:application/json;base64,${base64Encode(JSON.stringify(expectedMapFile2))}`)); // Act sut.transpiledLocationFor(mappedLocation({ fileName: 'file1.ts' })); @@ -96,9 +90,7 @@ describe('SourceMapper', () => { it('should cache source maps for future use when `transpiledLocationFor` is called', () => { // Arrange const expectedMapFile1 = { sources: ['file1.ts'] }; - transpiledFiles.push(textFile({ - name: 'file1.js', mutated: true, content: `// # sourceMappingURL=data:application/json;base64,${base64Encode(JSON.stringify(expectedMapFile1))}` - })); + transpiledFiles.push(new File('file1.js', `// # sourceMappingURL=data:application/json;base64,${base64Encode(JSON.stringify(expectedMapFile1))}`)); // Act sut.transpiledLocationFor(mappedLocation({ fileName: 'file1.ts' })); @@ -114,39 +106,48 @@ describe('SourceMapper', () => { }); it('should throw an error if source map file is a binary file', () => { - transpiledFiles.push(textFile({ - name: 'file1.js', mutated: true, content: '// # sourceMappingURL=file1.js.map' - })); - transpiledFiles.push(binaryFile({ - name: 'file1.js.map' - })); + transpiledFiles.push(new File('file.js', '// # sourceMappingURL=file1.js.map')); + transpiledFiles.push(new File('file1.js.map', Buffer.from(PNG_BASE64_ENCODED, 'base64'))); expect(() => sut.transpiledLocationFor(mappedLocation({ fileName: 'foobar' }))) - .throws(SourceMapError, `Source map file "file1.js.map" has the wrong file kind. "Binary" instead of "Text"${ERROR_POSTFIX}`); + .throws(SourceMapError, /^Source map file "file1.js.map" could not be parsed as json. Cannot analyse code coverage. Setting `coverageAnalysis: "off"` in your stryker.conf.js will prevent this error/); }); it('should throw an error if source map data url is not supported', () => { const expectedMapFile1 = { sources: ['file1.ts'] }; - transpiledFiles.push(textFile({ - name: 'file1.js', mutated: true, content: `// # sourceMappingURL=data:application/xml;base64,${base64Encode(JSON.stringify(expectedMapFile1))}` - })); + transpiledFiles.push(new File('file1.js', `// # sourceMappingURL=data:application/xml;base64,${base64Encode(JSON.stringify(expectedMapFile1))}`)); expect(() => sut.transpiledLocationFor(mappedLocation({ fileName: 'foobar' }))) .throws(SourceMapError, `Source map file for "file1.js" cannot be read. Data url "data:application/xml;base64" found, where "data:application/json;base64" was expected${ERROR_POSTFIX}`); }); it('should throw an error if source map file cannot be found', () => { - transpiledFiles.push(textFile({ - name: 'file1.js', mutated: true, content: `// # sourceMappingURL=file1.js.map` - })); + transpiledFiles.push(new File('file1.js', '// # sourceMappingURL=file1.js.map')); expect(() => sut.transpiledLocationFor(mappedLocation({ fileName: 'foobar' }))) .throws(SourceMapError, `Source map file "file1.js.map" (referenced by "file1.js") cannot be found in list of transpiled files${ERROR_POSTFIX}`); }); it('should throw an error if source map file url is not declared in a transpiled file', () => { - transpiledFiles.push(textFile({ - name: 'file1.js', mutated: true, content: `// # sourceMapping%%%=file1.js.map` - })); + transpiledFiles.push(new File('file1.js', `// # sourceMapping%%%=file1.js.map`)); expect(() => sut.transpiledLocationFor(mappedLocation({ fileName: 'foobar' }))) - .throws(SourceMapError, `No source map reference found in transpiled file "file1.js"${ERROR_POSTFIX}`); + .throws(SourceMapError, `Source map not found for "foobar"${ERROR_POSTFIX}`); + }); + + it('should not throw an error if one of the files is a binary file', () => { + const expectedMapFile1 = { sources: ['file1.ts'] }; + transpiledFiles.push(new File('file1.js', `// # sourceMappingURL=data:application/json;base64,${base64Encode(JSON.stringify(expectedMapFile1))}`)); + transpiledFiles.push(new File('foo.png', Buffer.from(PNG_BASE64_ENCODED, 'base64'))); + expect(sut.transpiledLocationFor(mappedLocation({ fileName: 'file1.ts' }))).deep.eq({ + fileName: 'file1.js', + location: { + end: { + column: 2, + line: 0 + }, + start: { + column: 2, + line: 0 + } + } + }); }); }); }); \ No newline at end of file diff --git a/packages/stryker/test/unit/transpiler/TranspilerFacadeSpec.ts b/packages/stryker/test/unit/transpiler/TranspilerFacadeSpec.ts index f75a17f351..b8cc71435a 100644 --- a/packages/stryker/test/unit/transpiler/TranspilerFacadeSpec.ts +++ b/packages/stryker/test/unit/transpiler/TranspilerFacadeSpec.ts @@ -1,8 +1,9 @@ import { expect } from 'chai'; import { Config } from 'stryker-api/config'; import TranspilerFacade from '../../../src/transpiler/TranspilerFacade'; -import { Transpiler, TranspilerFactory, TranspileResult } from 'stryker-api/transpile'; -import { file, mock, Mock, transpileResult } from '../../helpers/producers'; +import { Transpiler, TranspilerFactory } from 'stryker-api/transpile'; +import { mock, Mock } from '../../helpers/producers'; +import { File } from 'stryker-api/core'; describe('TranspilerFacade', () => { let createStub: sinon.SinonStub; @@ -19,11 +20,10 @@ describe('TranspilerFacade', () => { }); it('should return input when `transpile` is called', async () => { - const input = [file({ name: 'input' })]; - const result = await sut.transpile(input); + const input = [new File('input', '')]; + const outputFiles = await sut.transpile(input); expect(createStub).not.called; - expect(result.error).is.null; - expect(result.outputFiles).eq(input); + expect(outputFiles).eq(input); }); }); @@ -31,8 +31,8 @@ describe('TranspilerFacade', () => { let transpilerOne: Mock; let transpilerTwo: Mock; - let resultOne: TranspileResult; - let resultTwo: TranspileResult; + let resultFilesOne: ReadonlyArray; + let resultFilesTwo: ReadonlyArray; let config: Config; beforeEach(() => { @@ -40,13 +40,13 @@ describe('TranspilerFacade', () => { config.transpilers.push('transpiler-one', 'transpiler-two'); transpilerOne = mock(TranspilerFacade); transpilerTwo = mock(TranspilerFacade); - resultOne = transpileResult({ outputFiles: [file({ name: 'result-1' })] }); - resultTwo = transpileResult({ outputFiles: [file({ name: 'result-2' })] }); + resultFilesOne = [new File('result-1', '')]; + resultFilesTwo = [new File('result-2', '')]; createStub .withArgs('transpiler-one').returns(transpilerOne) .withArgs('transpiler-two').returns(transpilerTwo); - transpilerOne.transpile.returns(resultOne); - transpilerTwo.transpile.returns(resultTwo); + transpilerOne.transpile.resolves(resultFilesOne); + transpilerTwo.transpile.resolves(resultFilesTwo); }); it('should create two transpilers', () => { @@ -58,34 +58,28 @@ describe('TranspilerFacade', () => { it('should chain the transpilers when `transpile` is called', async () => { sut = new TranspilerFacade({ config, produceSourceMaps: true }); - const input = [file({ name: 'input' })]; - const result = await sut.transpile(input); - expect(result).eq(resultTwo); + const input = [new File('input', '')]; + const outputFiles = await sut.transpile(input); + expect(outputFiles).eq(resultFilesTwo); expect(transpilerOne.transpile).calledWith(input); - expect(transpilerTwo.transpile).calledWith(resultOne.outputFiles); + expect(transpilerTwo.transpile).calledWith(resultFilesOne); }); - it('should chain an additional transpiler when requested', async () => { - const additionalTranspiler = mock(TranspilerFacade); - const expectedResult = transpileResult({ outputFiles: [file({ name: 'result-3' })] }); - additionalTranspiler.transpile.returns(expectedResult); - const input = [file({ name: 'input' })]; - sut = new TranspilerFacade( - { config, produceSourceMaps: true }, - { name: 'someTranspiler', transpiler: additionalTranspiler } - ); - const output = await sut.transpile(input); - expect(output).eq(expectedResult); - expect(additionalTranspiler.transpile).calledWith(resultTwo.outputFiles); - }); - - it('should stop chaining if an error occurs during `transpile`', async () => { + // Arrange + transpilerOne.transpile.reset(); + const expectedError = new Error('an error'); + transpilerOne.transpile.rejects(expectedError); sut = new TranspilerFacade({ config, produceSourceMaps: true }); - const input = [file({ name: 'input' })]; - resultOne.error = 'an error'; - const result = await sut.transpile(input); - expect(result).eq(resultOne); + const input = [new File('input', '')]; + + // Act + const transpilePromise = sut.transpile(input); + + // Assert + await (expect(transpilePromise).rejectedWith('An error occurred in transpiler "transpiler-one". Inner error: Error: an error')); + + // Assert expect(transpilerOne.transpile).calledWith(input); expect(transpilerTwo.transpile).not.called; }); diff --git a/packages/stryker/test/unit/transpiler/coverageHooksSpec.ts b/packages/stryker/test/unit/transpiler/coverageHooksSpec.ts new file mode 100644 index 0000000000..b9f28ae08d --- /dev/null +++ b/packages/stryker/test/unit/transpiler/coverageHooksSpec.ts @@ -0,0 +1,23 @@ +import { expect } from 'chai'; +import * as coverageHooks from '../../../src/transpiler/coverageHooks'; +import { testFramework } from '../../helpers/producers'; + +describe('coveragePerTestHooks', () => { + + it('should use the coverage variable "__strykerCoverageCurrentTest__"', () => { + const actual = coverageHooks.coveragePerTestHooks(testFramework()); + expect(actual).to.contain('__strykerCoverageCurrentTest__'); + }); + + it('should use beforeEach and afterEach (test framework hooks)', () => { + const actual = coverageHooks.coveragePerTestHooks(testFramework()); + expect(actual).to.contain('beforeEach()'); + expect(actual).to.contain('afterEach()'); + }); + + it('should wrap all in a closure', () => { + const actual = coverageHooks.coveragePerTestHooks(testFramework()); + expect(actual).to.contain('(function (window) {'); + expect(actual).to.contain('})((Function(\'return this\'))());'); + }); +}); \ No newline at end of file diff --git a/packages/stryker/test/unit/utils/fileUtilsSpec.ts b/packages/stryker/test/unit/utils/fileUtilsSpec.ts index 97511e7d67..98f63f4da5 100644 --- a/packages/stryker/test/unit/utils/fileUtilsSpec.ts +++ b/packages/stryker/test/unit/utils/fileUtilsSpec.ts @@ -14,42 +14,4 @@ describe('fileUtils', () => { expect(fs.writeFile).to.have.been.calledWith('filename', 'data', 'utf8'); }); }); - - describe('isOnlineFile', () => { - describe('returns true', () => { - it('when provided with http://google.com', () => { - let url = 'http://google.com'; - - let result = fileUtils.isOnlineFile(url); - - expect(result).to.be.true; - }); - - it('when provided with https://google.com', () => { - let url = 'https://google.com'; - - let result = fileUtils.isOnlineFile(url); - - expect(result).to.be.true; - }); - }); - - describe('returns false', () => { - it('when provided with http:/google.com', () => { - let url = 'http:/google.com'; - - let result = fileUtils.isOnlineFile(url); - - expect(result).to.be.false; - }); - - it('when provided with google.com', () => { - let url = 'google.com'; - - let result = fileUtils.isOnlineFile(url); - - expect(result).to.be.false; - }); - }); - }); }); \ No newline at end of file