feat: schema for stryker configuration file

[bartekleon]

fixes: #1851

[bartekleon]

now, just by adding "$schema": "https://raw.githubusercontent.com/stryker-mutator/stryker/master/stryker.schema.json" should allow having nice autocompletion

[nicojs]

This is an awesome idea, I guess #1853 should be merged first. I love to see that you're creating small merge requests!

Next step can be to generate StrykerOptions.ts from the schema file. I'm using https://github.com/bcherny/json-schema-to-typescript for that in my day-to-day work.

[bartekleon]

I might your need @nicojs in that part :/ + this schema is only beta version. We still miss some properties I would like to add. But for now.it should be enough

[nicojs]

Awesome! I will add it when I have some time. Thanks!

[bartekleon]

"I love to see that you're creating small merge requests!", Nico, I know your experience with my PR's is 'humongous', but please spare me it ;D

I'll take a look at what I can improve before merging (I am looking mostly at babel plugins and excluding mutations right now)

[bartekleon]

@nicojs you may want to take a look. It's still buggy and some things should be changed but yea. It should be easier for you to help me now (I guess)

[bartekleon]

@nicojs I might need your help soon. For now, this schema is pretty accurate, but please take a look what is missing :/

below I'll paste how current (updated after each commit) Output would look like

[bartekleon]

/* tslint:disable */
/**
 * This file was automatically generated by json-schema-to-typescript.
 * DO NOT MODIFY IT BY HAND. Instead, modify the source JSONSchema file,
 * and run json-schema-to-typescript to regenerate this file.
 */

/**
 * JSON schema for the Stryker Mutator configuration file
 */
export interface StrykerOptions {
  /**
   * The 'allowConsoleColors' value indicates whether Stryker should use colors in console.
   */
  allowConsoleColors: true | false;
  /**
   * With 'coverageAnalysis' you specify which coverage analysis strategy you want to use.
   */
  coverageAnalysis: 'off' | 'all' | 'perTest';
  dashboard: DashboardOptions;
  /**
   * Set the log level that Stryker uses to write to the "stryker.log" file
   */
  fileLogLevel: 'off' | 'fatal' | 'error' | 'warn' | 'info' | 'debug' | 'trace';
  /**
   * With 'files', you can choose which files should be included in your test runner sandbox. This is normally not needed as it defaults to all files not ignored by git. Try it out yourself with this command: 'git ls-files --others --exclude-standard --cached --exclude .stryker-tmp.'
   */
  files?: string[];
  /**
   * Set the log level that Stryker uses to write to the console.
   */
  logLevel: 'off' | 'fatal' | 'error' | 'warn' | 'info' | 'debug' | 'trace';
  /**
   * Specifies the maximum number of concurrent test runners to spawn. Mutation testing is time consuming. By default, Stryker tries to make the most of your CPU, by spawning as many test runners as you have CPU cores.
   */
  maxConcurrentTestRunners: number;
  /**
   * With mutate you configure the subset of files to use for mutation testing. Generally speaking, these should be your own source files.
   */
  mutate: string[];
  /**
   * With mutator you configure which mutator plugin you want to use, and optionally, which mutation types to exclude from the test run.
   */
  mutator:
    | ('javascript' | 'typescript' | 'vue')
    | {
        name?: string;
        plugins?: string[];
        excludedMutations?: string[];
      };
  /**
   * With 'plugins', you can add additional Node modules for Stryker to load (or require). By default, all node_modules starting with @stryker-mutator/* will be loaded, so you would normally not need to specify this option. These modules should be installed right next to stryker. For a current list of plugins, you can consult 'npm' or 'stryker-mutator.io.'
   */
  plugins: string[];
  /**
   * With reporters, you can set the reporters for stryker to use.
   */
  reporters: string[];
  /**
   * The 'symlinkNodeModules' value indicates whether Stryker should create a symbolic link to your current node_modules directory in the sandbox directories. This makes running your tests by Stryker behave more like your would run the tests yourself in your project directory. Only disable this setting if you really know what you are doing.
   */
  symlinkNodeModules: true | false;
  /**
   * Choose a different temp dir that Stryker uses for mutation testing. This directory will contain copies of your source code during a mutation test run. It will be created if it not exists and is *entirely deleted* after a successful run, so change this with caution.
   */
  tempDirName: string;
  /**
   * Configure which test framework you are using. This option is not mandatory, as Stryker is test framework agnostic (it doesn't care what framework you use), However, it is required when coverageAnalysis is set to 'perTest', because Stryker needs to hook into the test framework in order to measure code coverage results per test and filter tests to run.
   */
  testFramework?: string;
  /**
   * With 'testRunner' you specify the test runner that Stryker uses to run your tests. The default value is command. The command runner runs a configurable bash/cmd command and bases the result on the exit code of that program (0 for success, otherwise failed). You can configure this command via the config file using the 'commandRunner: { command: 'npm run mocha' }'. It uses 'npm test' as the command by default.
   */
  testRunner: string;
  /**
   * Specify the thresholds for mutation score.
   */
  thresholds: {
    high: number;
    low: number;
    break: number | null;
  };
  /**
   * When Stryker is mutating code, it cannot determine indefinitely whether a code mutation results in an infinite loop (see Halting problem). In order to battle infinite loops, a test run gets killed after a certain period of time. This period is configurable with two settings: 'timeoutMS' and 'timeoutFactor'.
   */
  timeoutFactor: number;
  /**
   * When Stryker is mutating code, it cannot determine indefinitely whether a code mutation results in an infinite loop (see Halting problem). In order to battle infinite loops, a test run gets killed after a certain period of time. This period is configurable with two settings: 'timeoutMS' and 'timeoutFactor'.
   */
  timeoutMS: number;
  /**
   * With 'transpilers' you configure which transpiler plugins should transpile the code before it's executed. This is an array where the transpilers are called in the other of the array. This defaults to an empty array meaning no transpilation will be done.
   */
  transpilers: string[];
  [k: string]: any;
}
/**
 * This interface was referenced by `StrykerOptions`'s JSON-Schema
 * via the `definition` "dashboardOptions".
 */
export interface DashboardOptions {
  /**
   * Indicates which project to use if the "dashboard" reporter is enabled.
   */
  project?: string;
  /**
   * Indicates which version to use if the "dashboard" reporter is enabled.
   */
  version?: string;
  /**
   * Indicates which module to use if the "dashboard" reporter is enabled.
   */
  module?: string;
  /**
   * Indicates the base url of the stryker dashboard.
   */
  baseUrl: string;
  /**
   * Indicates wether to send a full report (inc. source code and mutant results) or only the mutation score.
   */
  reportType: 'full' | 'mutationScore';
}
/**
 * This interface was referenced by `StrykerOptions`'s JSON-Schema
 * via the `definition` "mutationScoreThresholds".
 */
export interface MutationScoreThresholds {
  high: number;
  low: number;
  break: number | null;
}
/**
 * Thresholds for mutation score. Disable a score with `null`.
 *
 * mutation score < break => exit build process with exit code 1. By default this is disabled (null).
 * mutation score < low => score is in danger zone, display in red.
 * mutation score < high >= low => score is in warning zone, display in yellow.
 * mutation score >= high => score is in awesome zone, display in green.
 *
 * This interface was referenced by `StrykerOptions`'s JSON-Schema
 * via the `definition` "mutatorDescriptor".
 */
export interface MutatorDescriptor {
  name: string;
  plugins: string[];
  excludedMutations: string[];
}

[nicojs]

Great job so far! Thanks for keeping this MR clean. I'll be working on it a little bit today to figure out how we should integrate it.

[nicojs]

Also interesting: https://github.com/SchemaStore/schemastore#contribute

[bartekleon]

So after creating schema we can publish it online to schema store, am I correct?

[nicojs]

Yes, that's the idea. I'm now doing small improvements on your branch in order to make it so we can generate the StrykerOptions.ts file from it. Will report back here when it's done.

[nicojs]

Just made it work. It took quite some work still since the out-of-the-box json-schema-to-typescript still has some strange side effects. I added some preprocessing, before generating the file.

I've remove the 'old' StrykerOptions files from source control. A npm run build or (ctrl+shift+b in vscode) will now first generate the typescript from json schema sources.

Thanks for the setup @kmdrGroch !

[nicojs]

@kmdrGroch , my friend: SchemaStore/schemastore#920

[bartekleon]

<3 good job us :D

We still need to declare $schema to make it works but well, better than nothing